{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c00-78812b6b",
   "metadata": {},
   "source": [
    "# Work queues with share groups — practical\n",
    "\n",
    "Forty fault tickets sit in a topic with a single partition. Four technicians in a **share group**\n",
    "work through them together. One technician crashes while holding a ticket, one works too slowly\n",
    "for the ticket's lock, and one ticket nobody can fix keeps bouncing.\n",
    "\n",
    "**Lab services:** kafka-1, kafka-2, kafka-3 · **Script:** `module-03/technician.py`\n",
    "· **Lesson:** `modules/03-consumers/04-work-queues-with-share-groups.md`"
   ]
  },
  {
   "cell_type": "code",
   "id": "c01-fb299609",
   "metadata": {},
   "source": [
    "# Setup: find the lab's shared helpers and check the lab is running\n",
    "import sys, pathlib\n",
    "lab_root = next(p for p in [pathlib.Path.cwd(), *pathlib.Path.cwd().parents] if (p / \"common\").is_dir())\n",
    "sys.path.insert(0, str(lab_root))\n",
    "\n",
    "import time\n",
    "from collections import Counter\n",
    "from common import settings, lab\n",
    "from common.scenario import Brightlane, to_json\n",
    "\n",
    "lab.require_kafka()\n",
    "lab.print_versions()"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c02-ca415dd7",
   "metadata": {},
   "source": [
    "## Step 1: The tickets, and the share group's settings\n",
    "The topic has 1 partition, so all four technicians share it. (With more partitions, the broker\n",
    "decides which technicians share which partition, and the run would differ every time.) Share\n",
    "groups are configured on the broker, per group: start at the earliest ticket (the default is\n",
    "`latest`), and lock each handed-out ticket for 15 seconds (default 30; 15 is the smallest the\n",
    "broker allows)."
   ]
  },
  {
   "cell_type": "code",
   "id": "c03-be443a2a",
   "metadata": {},
   "source": [
    "from confluent_kafka import Producer\n",
    "from confluent_kafka.admin import AdminClient, ConfigResource, ConfigEntry, AlterConfigOpType\n",
    "\n",
    "TOPIC, GROUP = \"m03l04-tickets\", \"m03l04-technicians\"\n",
    "admin = AdminClient({\"bootstrap.servers\": settings.KAFKA_BOOTSTRAP})\n",
    "lab.recreate_topic(admin, TOPIC, partitions=1)\n",
    "admin.delete_consumer_groups([GROUP])\n",
    "group_config = ConfigResource(\"group\", GROUP, incremental_configs=[\n",
    "    ConfigEntry(\"share.auto.offset.reset\", \"earliest\", incremental_operation=AlterConfigOpType.SET),\n",
    "    ConfigEntry(\"share.record.lock.duration.ms\", \"15000\", incremental_operation=AlterConfigOpType.SET)])\n",
    "admin.incremental_alter_configs([group_config])[group_config].result()"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c04-964b24d0",
   "metadata": {},
   "source": [
    "Tickets come in one at a time over the day, so each is written (and flushed) on its own. One\n",
    "extra ticket, `tkt-09999`, has an error code no technician knows."
   ]
  },
  {
   "cell_type": "code",
   "id": "c05-4175c861",
   "metadata": {},
   "source": [
    "tickets = Brightlane().fault_tickets(40)\n",
    "tickets.append({\"ticket_id\": \"tkt-09999\", \"charger_id\": \"BL-V02-01\", \"site_id\": \"site-V02\",\n",
    "                \"error_code\": \"E-UNKNOWN\", \"priority\": \"high\", \"opened_at\": tickets[-1][\"opened_at\"]})\n",
    "producer = Producer(settings.PRODUCER_DEFAULTS)\n",
    "for t in tickets:\n",
    "    producer.produce(TOPIC, key=t[\"ticket_id\"], value=to_json(t))\n",
    "    producer.flush(10)\n",
    "print(len(tickets), \"tickets:\", dict(Counter(t[\"error_code\"] for t in tickets)))"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c06-4ccc4559",
   "metadata": {},
   "source": [
    "## Step 2: Four technicians, one share group\n",
    "Each technician takes one ticket at a time. The script acknowledges each ticket explicitly:\n",
    "`ACCEPT` when fixed, `RELEASE` when it can't fix it. Nobody knows `E-UNKNOWN`. Ben also can't fix\n",
    "cable faults (`E-CABLE`). Cleo is slow: a ticket takes her 20 seconds. Dev is careful: 2 seconds.\n",
    "Ana is quick: 0.3 seconds."
   ]
  },
  {
   "cell_type": "code",
   "id": "c07-4948c61d",
   "metadata": {},
   "source": [
    "def technician(name, work_ms, skip=\"E-UNKNOWN\"):\n",
    "    return lab.start_script(\"module-03/technician.py\", \"--name\", name, \"--topic\", TOPIC, \"--group\", GROUP,\n",
    "                            \"--work-ms\", str(work_ms), \"--skip-codes\", skip, name=f\"m03l04-{name}\")\n",
    "\n",
    "techs = {\"ana\": technician(\"ana\", 300), \"ben\": technician(\"ben\", 300, \"E-UNKNOWN,E-CABLE\"),\n",
    "         \"cleo\": technician(\"cleo\", 20000), \"dev\": technician(\"dev\", 2000)}\n",
    "time.sleep(6)\n",
    "for name, t in techs.items():\n",
    "    print(f\"{name:<5} has taken {len(t.lines(' GOT ')):>2} tickets, fixed {len(t.lines(' DONE ')):>2}\")"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c08-643576d0",
   "metadata": {},
   "source": [
    "You should see all four technicians with tickets from one partition: a consumer group would have\n",
    "kept three of them idle. Ana has fixed the most; Cleo is still on her first. The broker's view of\n",
    "the members shows each one assigned the same partition:"
   ]
  },
  {
   "cell_type": "code",
   "id": "c09-b8fc37e5",
   "metadata": {},
   "source": [
    "print(lab.kafka_cli(\"kafka-share-groups.sh\", \"--bootstrap-server\", \"localhost:19092\",\n",
    "                    \"--describe\", \"--group\", GROUP, \"--members\"))"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c10-5bf14534",
   "metadata": {},
   "source": [
    "## Step 3: A technician's tablet dies\n",
    "Predict first: Dev's process is killed while he holds a ticket. What happens to that ticket?"
   ]
  },
  {
   "cell_type": "code",
   "id": "c11-cb8d2eae",
   "metadata": {},
   "source": [
    "holding = techs[\"dev\"].lines(\" GOT \")[-1]\n",
    "print(\"dev holds:\", holding.split(\" GOT \")[1])\n",
    "lab.kill_script(techs[\"dev\"])\n",
    "held = holding.split(\" GOT \")[1].split()[0]\n",
    "lab.wait_until(lambda: any(f\"GOT {held} \" in l and \"delivery=2\" in l for t in techs.values() for l in t.lines()),\n",
    "               timeout=40, what=f\"{held} to be handed out again\")\n",
    "print([l for t in techs.values() for l in t.lines() if f\"GOT {held} \" in l and \"delivery=2\" in l])"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c12-64f80d08",
   "metadata": {},
   "source": [
    "You should see Dev's ticket go to another technician with `delivery=2`, within a second or two.\n",
    "The broker released it as soon as Dev's connection dropped.\n",
    "\n",
    "## Step 4: Let the others finish\n",
    "Wait 30 seconds, stop everyone, and count per technician."
   ]
  },
  {
   "cell_type": "code",
   "id": "c13-f74d040e",
   "metadata": {},
   "source": [
    "time.sleep(30)\n",
    "for t in techs.values():\n",
    "    lab.stop_script(t)\n",
    "lab.table([(n, len(t.lines(\" GOT \")), len(t.lines(\" DONE \")), len(t.lines(\"RELEASED\")))\n",
    "           for n, t in techs.items()], [\"technician\", \"tickets taken\", \"fixed\", \"released\"])"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c14-3c61e52d",
   "metadata": {},
   "source": [
    "## Break it: the slow technician and the bouncing ticket\n",
    "Look at every ticket that was handed out more than once."
   ]
  },
  {
   "cell_type": "code",
   "id": "c15-7941ae07",
   "metadata": {},
   "source": [
    "for t in techs.values():\n",
    "    for line in t.lines():\n",
    "        if \" GOT \" in line and \"delivery=1 \" not in line:\n",
    "            print(line)\n",
    "print(\"Cleo:\", [l for l in techs[\"cleo\"].lines() if \" GOT \" in l or \" DONE \" in l])"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c16-0edafda6",
   "metadata": {},
   "source": [
    "Three things to find in this list:\n",
    "- Cleo's first ticket was handed to someone else after 15 seconds, the lock duration, while she\n",
    "  was still working on it. It was fixed twice.\n",
    "- `tkt-09999` went round with `delivery=2`, `3`, `4`, `5` and then stopped: after 5 deliveries\n",
    "  (`share.delivery.count.limit`) the broker archives a record. Nobody will see it again.\n",
    "- Ben's cable tickets were given straight back to Ben, because his client kept fetching while he\n",
    "  released them, until the delivery limit archived them unfixed. (The cable tickets Ana happened\n",
    "  to take first, she fixed.)\n",
    "\n",
    "Finally, the share group's state on the broker:"
   ]
  },
  {
   "cell_type": "code",
   "id": "c17-23715b46",
   "metadata": {},
   "source": [
    "print(lab.kafka_cli(\"kafka-share-groups.sh\", \"--bootstrap-server\", \"localhost:19092\",\n",
    "                    \"--describe\", \"--group\", GROUP))"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "c18-4d36f18f",
   "metadata": {},
   "source": [
    "## Clean up\n",
    "Delete the topic and the share group. If anything gets stuck, `docker compose down -v` in the\n",
    "`code/` folder resets the whole lab."
   ]
  },
  {
   "cell_type": "code",
   "id": "c19-80507cae",
   "metadata": {},
   "source": [
    "lab.delete_topics(admin, [TOPIC])\n",
    "admin.delete_consumer_groups([GROUP])\n",
    "print(\"cleaned up\")"
   ],
   "execution_count": null,
   "outputs": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
