Glossary
Acknowledgement — A share-group member's answer for a record: ACCEPT (done), RELEASE (give it back) or REJECT (archive it). First used in Work queues with share groups.
acks — The producer setting that says when a write counts as stored: 0 (never confirmed), 1 (the leader has it) or all (every in-sync replica has it; the default). First used in Durable writes.
ACL (access control list) — Kafka's permission rules. Each ACL binding allows or denies one principal one operation on one resource (a topic, group, transactional id or the cluster). First used in Authorization and quotas.
Active controller — The one controller in the quorum that makes decisions and writes them to the metadata log; the others copy it and stand by. First used in Brokers, controllers and partitions.
Active segment — The segment a partition is currently writing to; retention and compaction never touch it. First used in Segments, retention and deleting data.
Active-active — Two clusters that both serve clients and copy to each other. Harder than active-passive: every consumer must handle records from both sides. First used in Disaster recovery with MirrorMaker 2.
Active-passive — Two clusters where one serves the clients and the other only receives copies, until a failover makes it the active one. First used in Disaster recovery with MirrorMaker 2.
Advertised listener — The address a broker tells clients to use when they connect to it. It must be reachable from where the client runs. First used in Brokers, controllers and partitions.
Alert rule — A query plus a duration: Prometheus marks the alert pending while the query returns something, and firing once that has lasted the for duration. First used in Observability: metrics, lag and alerts.
Apache Kafka — A system that stores event logs on several servers and lets many programs write and read them at the same time. First used in Why Brightlane needs an event log.
Archived record — A record a share group will never deliver again, after a REJECT or too many deliveries. First used in Work queues with share groups.
Assign — Reading chosen partitions from offsets you pick yourself, without a group tracking your position. Compare subscribe. First used in Records, offsets and replay.
Assignor — The function that decides which member of a consumer group gets which partitions, such as range or cooperative-sticky. First used in Rebalances and the new group protocol.
At-least-once — Processing where a crash can repeat records but never skip them: commit after processing. First used in Committing offsets.
At-most-once — Processing where a crash can skip records but never repeat them: commit before processing. First used in Committing offsets.
Authorization — Deciding what an authenticated principal may do. Kafka's authorizer checks every request against the ACLs. First used in Authorization and quotas.
Auto-commit — The client committing offsets in the background, every 5 seconds by default (enable.auto.commit). The Python client also marks records as done as soon as it hands them to you (enable.auto.offset.store). First used in Committing offsets.
auto.offset.reset — The consumer setting that says where a group with no stored position
starts: earliest (the oldest record) or latest (only new records; the Python client's default).
First used in Why Brightlane needs an event log.
Avro — A compact binary data format whose schema is written in JSON; values are stored without field names, in schema order. First used in Schemas and the Schema Registry.
Backoff — Waiting longer after each failed retry, for example 0.5, 1, 2 seconds. First used in Poison pills and dead-letter topics.
Batch — A group of records for one partition that the producer sends, compresses and the broker stores together. First used in Batching, compression and throughput.
batch.size — The largest batch, in bytes, the producer builds before sending it: 1,000,000 by default in the Python client, 16 KB in Java. First used in Batching, compression and throughput.
Bootstrap servers — The broker addresses a client contacts first, only to fetch the cluster's metadata. List several. First used in Brokers, controllers and partitions.
Broker — A Kafka server process that stores partitions on its disk and answers the clients that write and read them. First used in Brokers, controllers and partitions.
Certificate — A file with a server's names and public key, signed by a certificate authority. A broker shows it during the TLS handshake. First used in Encryption and authentication.
Certificate authority (CA) — A key pair whose signature clients agree to trust: a certificate it signed is accepted as genuine. Clients point ssl.ca.location at its certificate. First used in Encryption and authentication.
Change data capture (CDC) — Turning every change in a database's tables into events, read from the database's own log. First used in Change data capture from PostgreSQL.
Changelog topic — A compacted topic that records every change of a state store, so the store can be rebuilt. First used in State: turning a stream into a table.
Checkpoint — A saved copy of a processor's state and how far its changelogs went, for a fast restart. First used in State: turning a stream into a table.
Claim check — A pattern for large payloads: store the file in object storage and send an event that carries a link to it. First used in The production-readiness review.
Clean leader election — Choosing a partition's new leader from its in-sync replicas, so no acknowledged record is lost. First used in Leaders, followers and failover.
Cleanup policy — A topic's cleanup.policy: delete removes old segments by age or size, compact keeps the latest record per key, or both. First used in Segments, retention and deleting data.
Client quota — A limit on how much of the brokers a principal or client id may use: bytes per second written or read, request time, or topic changes. Each broker enforces it on its own. First used in Authorization and quotas.
Cluster — A group of brokers and controllers that work together as one Kafka system. First used in Brokers, controllers and partitions.
Co-partitioning — Two topics keyed the same way, with the same partition count and partitioner, so a key is in the same partition of both. First used in State: turning a stream into a table.
Combined mode — Running a broker and a controller in the same process. Fine for a laptop lab; production runs controllers separately. First used in Brokers, controllers and partitions.
Committed offset — The offset of the next record a consumer group should process in a partition, saved in Kafka's __consumer_offsets topic. First used in Committing offsets.
Compacted topic — A topic with cleanup.policy=compact, which keeps at least the latest record for every key. First used in Compacted topics.
Compatibility mode — A subject's rule for new versions: BACKWARD (the default), FORWARD, FULL, their _TRANSITIVE variants, or NONE. First used in Evolving a schema.
Compression — Shrinking each batch with a codec (gzip, snappy, lz4 or zstd) set by the producer's compression.type. Brokers store and replicate it compressed; consumers decompress it. First used in Batching, compression and throughput.
Config provider — A plug-in that fills in a configuration value at start-up, for example from an environment variable or a file, so secrets stay out of configuration files. First used in Encryption and authentication.
Connector — A Kafka Connect plug-in instance, configured through the REST API, that moves data from a system (source) or to one (sink). First used in Change data capture from PostgreSQL.
Consume-transform-produce — The usual shape of a stream processor: read records, compute, write records, then save the position. First used in Exactly-once billing with transactions.
Consumer — A program that reads records from a topic. First used in Why Brightlane needs an event log.
Consumer group — A shared name (group.id) for consumers that read a topic together. Kafka
stores one position per group, so different groups read independently. First used in
Why Brightlane needs an event log.
Consumer protocol (KIP-848) — The group protocol, generally available since Kafka 4.0, where the broker computes assignments and changes them incrementally (group.protocol=consumer). First used in Rebalances and the new group protocol.
Controlled shutdown — A clean broker stop: the broker asks the controller to move its leaderships before it exits, so clients barely notice. First used in Leaders, followers and failover.
Controller — A node that manages the cluster's metadata: which brokers are alive, which topics exist, and which broker leads each partition. First used in Brokers, controllers and partitions.
Converter — The Kafka Connect component that turns records into bytes and back, such as AvroConverter. First used in Change data capture from PostgreSQL.
Cooperative rebalance — A rebalance where members keep the partitions that don't move; only the moving ones are revoked and reassigned. First used in Rebalances and the new group protocol.
CreateTime — The default record timestamp: set by the producer when it creates the record. Compare LogAppendTime. First used in Records, offsets and replay.
Crypto-shredding — Encrypting each person's data with their own key and deleting the key to make every copy unreadable. First used in Segments, retention and deleting data.
Dead-letter topic — A topic where a consumer parks records it can't process, with headers saying why, so it can carry on (also called a DLQ). First used in Poison pills and dead-letter topics.
Debezium — An open-source family of Kafka Connect connectors for change data capture. First used in Change data capture from PostgreSQL.
Default value — The value a reader uses for a field the writer's data doesn't have; it makes adding a field compatible. First used in Evolving a schema.
Delivery count — How many times a share group has handed out a record. At share.delivery.count.limit (5 by default) the record is archived. First used in Work queues with share groups.
Delivery report — The callback the producer runs once per record, from poll() or flush(), with either an error or the partition and offset where the record was stored. First used in Durable writes.
Delivery timeout — How long the producer keeps retrying a record before it reports failure: message.timeout.ms in the Python client (5 minutes by default), delivery.timeout.ms in Java. First used in Durable writes.
Deny by default — With allow.everyone.if.no.acl.found=false, the broker refuses whatever no ACL allows. First used in Authorization and quotas.
Dirty ratio — The share of a compacted partition written since its last cleaning; the cleaner starts at min.cleanable.dirty.ratio (0.5). First used in Compacted topics.
Diskless topics — A proposed kind of topic (KIP-1150) whose data goes straight to object storage, without broker disks or broker-to-broker replication. Not in a release yet. First used in The production-readiness review.
Drift — A difference between the cluster and the configuration file that describes it, usually from a change made by hand. First used in Day-2 operations: change without downtime.
Eager rebalance — A classic-protocol rebalance where every member gives up all its partitions until the new assignment arrives. First used in Rebalances and the new group protocol.
Eligible leader replicas (ELR) — Replicas that left the ISR while it was below min.insync.replicas but still have every acknowledged record, so they can safely become leader (Kafka 4.1+). First used in Leaders, followers and failover.
End offset — The offset the next record written to a partition will get; one more than the last stored record's offset. First used in Records, offsets and replay.
Enrichment join — Adding data from a table to each event of a stream, such as the tariff for a session's site. First used in State: turning a stream into a table.
Error tolerance — Kafka Connect's errors.tolerance: none (fail the task at the first bad record, the default) or all (skip it, or park it in a dead-letter topic). First used in Sinks, errors and dead letters in Connect.
Event — A record of something that happened, stated in the past tense and never changed afterwards, such as "session ses-7f3a19 stopped at 18:09". First used in Why Brightlane needs an event log.
Event log — An ordered list of events. New events are only added at the end, and stored events are never edited. First used in Why Brightlane needs an event log.
Event time — When an event happened, as recorded in the event itself. First used in Event time and windows.
Exactly-once — Processing where every record has its effect once, even after crashes: the result and the position are saved together, atomically. First used in Committing offsets.
Failback — Moving clients back from the DR cluster to the primary after it's repaired: a planned move at a quiet moment. First used in Disaster recovery with MirrorMaker 2.
Failover — Moving producers and consumers to another cluster when the one they use fails. First used in Disaster recovery with MirrorMaker 2.
Failure domain — A set of things that fail together: a disk, a broker, a rack, a data centre, a cloud region. First used in Disaster recovery with MirrorMaker 2.
Fake — In a test, a small stand-in object that behaves like a real client and records what it was asked to do. First used in Testing Kafka applications.
Feature version — A cluster-wide level, such as metadata.version, that switches on new formats and behaviour once every broker runs the new software. Raised with kafka-features.sh upgrade. First used in Day-2 operations: change without downtime.
Fenced broker — A broker the controller treats as gone because it sent no heartbeat for broker.session.timeout.ms (9 s): it leaves every ISR and loses its leaderships. First used in Leaders, followers and failover.
Fencing — Rejecting an older instance after a newer one with the same identity has started, using an epoch counter. First used in Exactly-once billing with transactions.
Fetch request — The request a consumer or a follower sends to a leader: "give me the records from offset N". First used in Leaders, followers and failover.
Follower — A replica that copies new records from its partition's leader, ready to take over if the leader's broker fails. First used in Brokers, controllers and partitions.
Grace period — How long after a window's end a processor still accepts its events, measured in stream time. First used in Event time and windows.
Group coordinator — The broker that keeps a consumer group's list of members, their partitions and the group's committed offsets. First used in Consumer groups share the work.
Header — An optional name–value pair on a record, for technical metadata such as a trace id. First used in Records, offsets and replay.
Heartbeat — A small message a group member sends to the coordinator every few seconds to show it's alive. First used in Consumer groups share the work.
Heartbeat (CDC) — A periodic write a Debezium connector makes to a captured table, so it can confirm its position and free WAL. First used in Change data capture from PostgreSQL.
High watermark — The offset up to which all in-sync replicas of a partition have the records. Consumers can read only up to it. First used in Durable writes.
Hostname verification — The client's check that the broker's certificate lists the address it connected to. On by default in the Python client since librdkafka 2.0. First used in Encryption and authentication.
Hot key — A key that carries a large share of the traffic, so the one partition it maps to becomes a bottleneck. First used in Keys decide order.
Idempotent processing — Processing that has the same effect whether a record is handled once or several times, for example an insert that ignores a session already charged. First used in Committing offsets.
Idempotent producer — A producer that numbers its batches so the broker can drop retried duplicates and keep order. On by default in Java, off by default in the Python client (enable.idempotence). First used in Durable writes.
In-sync replica (ISR) — A replica that is fully caught up with its partition's leader. The ISR list of a partition names all of them, the leader included. First used in Brokers, controllers and partitions.
Integration test — A test that runs code against the real systems it talks to, here a Kafka broker in a container. First used in Testing Kafka applications.
isolation.level — A consumer setting: read_committed returns only committed transactional records (the Python default), read_uncommitted returns everything. First used in Exactly-once billing with transactions.
JMX — Java Management Extensions: Java's built-in way to expose a program's internal measurements. Kafka brokers report their metrics through it. First used in Observability: metrics, lag and alerts.
JMX exporter — A Java agent that runs inside a broker and serves its JMX values as Prometheus metrics over HTTP. First used in Observability: metrics, lag and alerts.
JSON Schema — A schema language that checks JSON data; the records stay JSON on the wire. First used in Schemas and the Schema Registry.
Kafka Connect — The part of Apache Kafka that moves data between Kafka and other systems with configured connectors instead of code. First used in Change data capture from PostgreSQL.
Key — Optional bytes on a record that say what the record is about, such as a charger id. Records with the same key go to the same partition. First used in Records, offsets and replay.
KIP — Kafka Improvement Proposal: the public design document for a change to Kafka, such as KIP-848 or KIP-932. First used in Rebalances and the new group protocol.
KMinion — An open-source exporter that reads consumer groups' committed offsets and topics' end offsets from a cluster, and serves lag as Prometheus metrics. First used in Observability: metrics, lag and alerts.
KRaft — Kafka's built-in agreement protocol (based on Raft) that its controllers use to keep one consistent copy of the metadata. It replaced ZooKeeper, which Kafka 4.0 removed. First used in Brokers, controllers and partitions.
Lag — How far a consumer group is behind on a partition: the end offset minus the committed offset. First used in Consumer groups share the work.
Lag in seconds — How far behind a consumer is in time: the age of the oldest event it hasn't processed, or an estimate such as records behind ÷ records arriving per second. First used in Observability: metrics, lag and alerts.
Last stable offset — The offset up to which no transaction is still open; read_committed consumers read only up to it. First used in Exactly-once billing with transactions.
Late event — An event that arrives after its window has been reported. First used in Event time and windows.
Latency percentile — The time within which a given share of records completed: p50 is the median, p99 the time 99% of records beat. Better than an average for spotting slow tails. First used in Batching, compression and throughput.
Leader — The one replica of a partition that producers write to and consumers read from. First used in Brokers, controllers and partitions.
Least privilege — Giving each service exactly the permissions its job needs, and nothing more. First used in Authorization and quotas.
linger.ms — How long the producer waits for more records before sending a batch: 5 ms by default in the Python client and, since Kafka 4.0, in Java. First used in Batching, compression and throughput.
Listener — A network port a broker accepts connections on, with a name and a security protocol. First used in Encryption and authentication.
Log cleaner — The broker threads that compact partitions, removing records that have a newer record with the same key. First used in Compacted topics.
Log start offset — The first offset still readable in a partition; retention and delete_records() move it forward. First used in Segments, retention and deleting data.
LogAppendTime — A topic setting where the broker overwrites each record's timestamp with its own clock when it stores the record. First used in Records, offsets and replay.
Logical type — An Avro annotation that gives a plain type a meaning, such as timestamp-millis on a long. First used in Schemas and the Schema Registry.
max.poll.interval.ms — The longest a consumer may go between calls to consume() or poll() before it leaves its group as stuck; 5 minutes by default. First used in Rebalances and the new group protocol.
Metadata — The cluster's facts about itself: its brokers, topics, partitions, leaders and settings. First used in Brokers, controllers and partitions.
min.insync.replicas — A topic setting: the fewest in-sync replicas that must exist for an acks=all write to be accepted. 2 is the usual value with 3 replicas. First used in Durable writes.
Mirrored clusters — Independent clusters, usually at different sites, with a tool such as MirrorMaker 2 copying topics between them in the background. First used in Disaster recovery with MirrorMaker 2.
MirrorMaker 2 — Apache Kafka's tool for copying topics and consumer offsets between clusters, built from Kafka Connect connectors. MirrorMaker 1 was removed in Kafka 4.2. First used in Disaster recovery with MirrorMaker 2.
Mutual TLS — TLS in which the client also shows a certificate; the certificate's name becomes the client's identity. First used in Encryption and authentication.
Offset — A record's position in its partition: 0 for the first record, then 1, 2 and so on, in arrival order. Offsets are never reused. First used in Records, offsets and replay.
Offset sync — A pair MirrorMaker 2 records: this offset in the original topic is that offset in the copy. Offset translation uses the last sync at or before a committed offset. First used in Disaster recovery with MirrorMaker 2.
Outbox pattern — Writing an event into an outbox table in the same database transaction as the change it describes, and publishing it with change data capture. It avoids writing to a database and to Kafka separately. First used in The production-readiness review.
Page cache — The memory where the operating system keeps recently written and read file data. Consumers that keep up are served from it, without reading the disk. First used in Encryption and authentication.
Partition — One ordered log inside a topic, with its own offsets. A topic's partitions are spread over the brokers. First used in Brokers, controllers and partitions.
Partition reassignment — Moving partition replicas between brokers, with kafka-reassign-partitions.sh and a plan in JSON. New replicas copy the data, join the ISR, then the old ones are deleted. First used in Day-2 operations: change without downtime.
Partitioner — The part of the producer that picks each record's partition: a hash of the key modulo the partition count, or a spread when there is no key. Java uses the murmur2 hash; the Python client defaults to CRC32 unless you set partitioner=murmur2_random. First used in Keys decide order.
Poison pill — A record a consumer can never process; without a plan it stops the partition behind it. First used in Poison pills and dead-letter topics.
Poll loop — The main loop of a consumer: call consume() or poll() again and again, and handle what comes back. First used in Consumer groups share the work.
Preferred leader — The first replica in a partition's replica list; Kafka spreads them over the brokers and moves leadership back to them after failovers. First used in Leaders, followers and failover.
Principal — The identity a broker gives a connection after login, such as User:svc-billing. ACLs and quotas refer to principals. First used in Encryption and authentication.
Processing time — When an event reaches the processor. First used in Event time and windows.
Produce request — One network request from a producer to a broker, carrying the waiting batches for the partitions that broker leads. First used in Batching, compression and throughput.
Producer — A program that writes records to a topic. First used in Why Brightlane needs an event log.
Producer id — A number the cluster gives each idempotent producer. With the batch sequence numbers it lets the broker recognise duplicates. First used in Durable writes.
Prometheus — An open-source monitoring system that scrapes metrics from its targets, stores them as time series, and evaluates alert rules. First used in Observability: metrics, lag and alerts.
Protobuf — Protocol Buffers: a binary data format with numbered fields, an alternative to Avro. First used in Schemas and the Schema Registry.
Publication — A PostgreSQL list of tables whose changes are streamed to logical replication readers such as Debezium. First used in Change data capture from PostgreSQL.
Queue — A messaging system that hands each message to one receiver and deletes it once it is handled, such as RabbitMQ or Amazon SQS. First used in Why Brightlane needs an event log.
Quorum — The set of controllers that vote on metadata changes. A change is decided when a majority has stored it. First used in Brokers, controllers and partitions.
Reader schema — The schema a consumer asks for; Avro translates each record from the writer schema into it. First used in Evolving a schema.
Readiness check — A program that checks a release and the running system against a production checklist, and exits with an error if anything fails, so a pipeline can stop the release. First used in The production-readiness review.
Rebalance — The group redistributing its partitions among its members, after a member joins or leaves or the partitions change. First used in Consumer groups share the work.
Record — One stored event in Kafka: a key, a value, headers and a timestamp, plus the partition and offset the broker adds. First used in Records, offsets and replay.
Record lock — The time a share-group member may hold a record it has acquired before the broker hands it to someone else (share.record.lock.duration.ms, 30 s by default). First used in Work queues with share groups.
Remote topic — A topic that MirrorMaker 2 copied from another cluster. By default its name starts with the source cluster's name, as in primary.charging.sessions. First used in Disaster recovery with MirrorMaker 2.
Repartition — Writing a stream again under a new key to an intermediate topic, so it can be grouped or joined by that key. First used in State: turning a stream into a table.
Replay — Reading records again from an earlier position, for example after fixing a bug. First used in Why Brightlane needs an event log.
Replica — One copy of a partition, stored on one broker. First used in Brokers, controllers and partitions.
Replication factor — How many replicas each partition of a topic has; 3 is the usual choice. First used in Brokers, controllers and partitions.
Replication slot — PostgreSQL's record of how far a reader of the WAL has confirmed; the database keeps all WAL after it. First used in Change data capture from PostgreSQL.
Replication throttle — A bytes-per-second limit on the copying that a partition reassignment causes, so it doesn't slow down clients. First used in Day-2 operations: change without downtime.
Resource pattern — How an ACL names its resources: LITERAL (exactly this name) or PREFIXED (every name that starts with it). First used in Authorization and quotas.
Retention — How long a topic keeps records before deleting them; 7 days by default. First used in Why Brightlane needs an event log.
Rolling restart — Restarting brokers one at a time, waiting until every partition is fully replicated before the next, so the cluster keeps serving. First used in Day-2 operations: change without downtime.
RPO — Recovery point objective: how much recent data you may lose when something fails, given as time. First used in Disaster recovery with MirrorMaker 2.
RTO — Recovery time objective: how long it may take until the service runs again after a failure. First used in Disaster recovery with MirrorMaker 2.
Runbook — The written first steps for one alert: what to check, and what each answer means. First used in Observability: metrics, lag and alerts.
SASL — Simple Authentication and Security Layer: a standard way to add a login to a network protocol. Kafka's mechanisms are SCRAM, OAUTHBEARER, GSSAPI and PLAIN. First used in Encryption and authentication.
Schema — A formal description of a record: its fields, their types and their defaults. First used in Schemas and the Schema Registry.
Schema id — The number the registry gives a schema; every record carries it in its first bytes. First used in Schemas and the Schema Registry.
Schema Registry — A service that stores schemas, gives each an id, and checks new versions for compatibility. First used in Schemas and the Schema Registry.
SCRAM — A SASL mechanism that checks a user name and password without sending the password; the cluster stores only salted hashes. First used in Encryption and authentication.
Scrape — Prometheus fetching a target's page of metrics over HTTP, every few seconds. First used in Observability: metrics, lag and alerts.
Security protocol — How a listener protects its connections: PLAINTEXT (not at all), SSL (TLS), SASL_PLAINTEXT (a login) or SASL_SSL (TLS and a login). First used in Encryption and authentication.
Segment — One file of a partition's log on disk, named after its first offset; Kafka appends to the newest one only. First used in Segments, retention and deleting data.
Sequence number — The number an idempotent producer gives each batch it sends to a partition, counting up from 0. First used in Durable writes.
Serialize — Turn an object, such as a Python dict, into bytes so it can be stored or sent; the reverse is deserialize. First used in Records, offsets and replay.
session.timeout.ms — How long a member may go without a heartbeat before the group presumes it dead and moves its partitions; 45 seconds by default. First used in Consumer groups share the work.
Share group — A group whose members share records one by one; the broker assigns each partition to one or more members, and each record goes to one of them (KIP-932, Kafka 4.2+). First used in Work queues with share groups.
Single message transform (SMT) — A small change Kafka Connect applies to each record on its way, such as renaming its topic. First used in Change data capture from PostgreSQL.
Sink connector — A connector that reads from Kafka and writes to another system. First used in Change data capture from PostgreSQL.
Snapshot — A CDC connector's first read of every existing row, before it streams changes. First used in Change data capture from PostgreSQL.
Source connector — A connector that reads from another system and writes to Kafka. First used in Change data capture from PostgreSQL.
State store — A stream processor's local storage for state such as running totals, in memory or on disk. First used in State: turning a stream into a table.
Static membership — Giving each group member a fixed group.instance.id, so it can restart within its session timeout without a rebalance. First used in Rebalances and the new group protocol.
Stream time — The latest event time a stream processor has seen; it only moves forward and decides when windows close. First used in Event time and windows.
Stream-table duality — A table is a stream of changes reduced to the latest value per key; a table's changes form a stream. First used in State: turning a stream into a table.
Stretched cluster — One Kafka cluster with brokers and controllers in three nearby sites, so a site can fail without losing acknowledged data. First used in Disaster recovery with MirrorMaker 2.
Strimzi — An open-source Kubernetes operator that runs Kafka clusters, topics and users from Kubernetes resources. First used in The production-readiness review.
Subject — A named list of versions of one schema in the registry, by default <topic>-value or <topic>-key. First used in Schemas and the Schema Registry.
Subscribe — Reading a topic as a member of a consumer group, which hands you partitions and remembers your position. Compare assign. First used in Records, offsets and replay.
Super user — A principal listed in the broker's super.users; every permission check allows it. First used in Authorization and quotas.
Task — The unit of a connector that actually moves records; a connector runs up to tasks.max of them. First used in Change data capture from PostgreSQL.
Testcontainers — A library that starts Docker containers for a test run, gives the tests their addresses, and removes them afterwards. First used in Testing Kafka applications.
Throttling — How a broker enforces a quota: it tells the client how long to pause (the throttle time) and stops reading from its connection for that long. Nothing is rejected. First used in Authorization and quotas.
Throughput — How much a system handles per second, in records or bytes. First used in Batching, compression and throughput.
Tiered storage — Keeping a topic's older segments in object storage such as S3 while the brokers keep only recent ones on disk (Kafka 3.9+). First used in Segments, retention and deleting data.
Time series — The history of one metric with one set of labels: a value for each point in time. First used in Observability: metrics, lag and alerts.
Timestamp — The time stored with a record, in milliseconds since 1970: either CreateTime or LogAppendTime. First used in Records, offsets and replay.
TLS — Transport Layer Security: the standard way to encrypt a network connection and prove the server's identity with a certificate. Kafka's settings still call it SSL. First used in Encryption and authentication.
Tombstone — A record with a key and no value; in a compacted topic it deletes the key, and stays for delete.retention.ms. First used in Compacted topics.
Topic — A named event log in Kafka, such as charging.sessions, made of one or more partitions.
First used in Why Brightlane needs an event log.
Topics as code — Keeping topics' partitions, replication factor and settings in a reviewed file, and applying the difference to the cluster with a tool. First used in Day-2 operations: change without downtime.
Transaction — A group of writes to several partitions, plus consumer offsets, that become visible together or not at all. First used in Exactly-once billing with transactions.
Transaction coordinator — The broker that manages a transactional producer's transactions, kept in __transaction_state. First used in Exactly-once billing with transactions.
Transaction marker — A control record the coordinator writes into each partition of a transaction when it commits or aborts. First used in Exactly-once billing with transactions.
transactional.id — A producer's stable identity for transactions; a new instance with the same id fences the old one. First used in Exactly-once billing with transactions.
Tumbling window — A fixed-size time window, back to back with the next, without overlap. First used in Event time and windows.
Unclean leader election — Choosing an out-of-sync replica as leader when no in-sync one is left (unclean.leader.election.enable); records it lacks are lost. First used in Leaders, followers and failover.
Under-replicated partition — A partition whose ISR is smaller than its replica list: a copy is missing or behind. First used in Leaders, followers and failover.
Unit test — A test of one piece of logic on its own, with no network or broker: milliseconds per test. First used in Testing Kafka applications.
Upsert — Insert a row for a new key, update it for a known one; repeated writes leave the same result. First used in Sinks, errors and dead letters in Connect.
Value — The main content of a record, as bytes that Kafka never reads or checks. First used in Records, offsets and replay.
Wire format — The layout of a registry-serialized record: a magic byte 0, the 4-byte schema id, then the data. First used in Schemas and the Schema Registry.
Work queue — A list of independent jobs, each to be done once by whichever worker is free; order doesn't matter. First used in Work queues with share groups.
Worker — A Kafka Connect process; workers form a group and run the connectors' tasks. First used in Change data capture from PostgreSQL.
Write-ahead log (WAL) — PostgreSQL's log of every change, written before the tables themselves; CDC reads it. First used in Change data capture from PostgreSQL.
Writer schema — The schema a record was written with; its id is in the record. First used in Evolving a schema.