Apache Kafka and Spring Kafka
Kafka is the spine between Telemetry and the Judge: about 25,000 events per second flow in, flags flow out, and the Judge even stores its only state in a compacted topic. This module builds Kafka up from "an append-only log" to producers, consumer groups, offsets, replication, compaction, transactions and error handling with dead-letter topics, all tied to your configuration files.
By the end you can answer
- "What is Kafka? How is it different from a traditional message queue?"
- "What are partitions for? How is ordering guaranteed?"
- "How do consumer groups and rebalancing work? What is consumer lag?"
- "Explain acks, idempotent producers and transactions. What does
read_committeddo?" - "At-most-once, at-least-once, exactly-once: which did you use where?"
- "Retention vs compaction? How can a topic be a database?"
- "How did you handle poison messages and outages in consumers?"
1. The big idea: a distributed, append-only log
A log here means an ordered, append-only sequence of records. New records are added at the end and never changed. Each record gets a number, its offset. Readers don't remove anything; each reader just remembers how far it has read.
topic "match-events", partition 3:
offset: 0 1 2 3 4 5 6 ... (append →)
[shot] [pos] [shot] [kill] [shot] [pos] [shot]
▲ ▲
detector group's committed offset verifier is here (reads independently)
A traditional queue is a to-do pile: someone takes a task and it's gone. Kafka is a newspaper archive: every issue is kept (for a configured time), and every reader has a bookmark. The detector, the verifier and a brand-new analytics job can all read the same issues at their own pace, and anyone can re-read from an older bookmark ("replay").
| Kafka (log) | Classic queue (RabbitMQ, SQS) | |
|---|---|---|
| After reading | The record stays; the reader advances its offset | The message is acknowledged and deleted |
| Many independent readers | Natural (each group has its own offsets) | Needs fan-out exchanges or one queue per reader |
| Replay history | Yes (reset offsets) | No |
| Ordering | Per partition | Usually per queue, weakened by multiple consumers or redelivery |
| Throughput | Very high (sequential disk I/O, batching, zero-copy) | High, with more per-message work |
| Per-message routing, priorities, delays | Limited | Rich |
2. Core vocabulary
- Broker: a Kafka server. A cluster is several brokers. Telemetry runs one (
apache/kafkain Compose). - Topic: a named stream of records, such as
match-eventsorsuspicious-players. - Partition: a topic is split into partitions, each an independent ordered log that can live on a different broker.
- Record: key (optional), value, timestamp, headers. In your projects keys are player ids (strings) and values are UTF-8 JSON.
- Offset: the position of a record within its partition.
- Producer writes; consumer reads; a consumer group is a set of consumers sharing the work of one subscription.
3. Partitions, keys and ordering
Partitions exist for two reasons: parallelism (different partitions can be written and read by different machines at once) and ordering (order is guaranteed within a partition, and only there).
With a key, the default partitioner computes murmur2(keyBytes) mod numPartitions, so the same key always lands in the same partition. Every event of player X goes to one partition, in the order it was produced, and one consumer sees all of them in order. That's exactly what per-player windows and per-player sequence checks need.
| Topic | Partitions | Cleanup | Why that shape |
|---|---|---|---|
match-events | 6 | delete, 24 h | High volume; 6 partitions allow up to 6 parallel detector tasks |
player-profiles | 6 | compact | A table (latest profile per player); co-partitioned with flags for the join |
suspicious-players | 6 | delete, 7 d | Output flags, keyed by player |
integrity-stats | 6 | compact | Latest count, duplicates and gaps per player |
match-events.DLT | 6 | delete, 7 d | Bad events, same keying |
match-summaries / .DLT | 3 | delete, 7 d | Low volume, one per finished match |
cheater-ground-truth, simulation-manifest | 1 | compact | Small reference data read whole by the verifier |
judge-verdicts (Judge) | 6 | compact | The Judge's only state: latest verdict per flag |
- Max consumer parallelism = number of partitions. A 7th consumer in a group for a 6-partition topic sits idle.
- Adding partitions later remaps keys (
hash mod 6≠hash mod 8), breaking per-key ordering across the change and breaking co-partitioning. Choose counts up front. - Hot partitions: if one key dominates, its partition is overloaded no matter how many partitions exist.
- Telemetry's rules say: use the default partitioner and never write a custom one, so producers in any language agree on placement.
4. Producers: batching, acks and idempotence
spring.kafka.producer:
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.apache.kafka.common.serialization.ByteArraySerializer # bytes, so garbage can be injected on purpose
acks: all
batch-size: 65536 # up to 64 KB per partition batch
compression-type: lz4
properties:
enable.idempotence: true
linger.ms: 10 # wait up to 10 ms to fill a batch
max.in.flight.requests.per.connection: 5
delivery.timeout.ms: 120000 # give up after 2 minutes of retries
Throughput: batching and compression
send() doesn't hit the network immediately. Records are appended to an in-memory batch per partition, which is sent when it's full (batch.size) or after linger.ms. Bigger batches mean fewer requests and better compression (lz4 is fast). 25,000 small JSON events per second is easy for one producer this way. The trade-off is a few milliseconds of added latency.
Durability: acks
acks | The producer's send succeeds when | Risk |
|---|---|---|
| 0 | It's on the wire | Silent loss |
| 1 | The partition leader wrote it | Loss if the leader dies before followers copy it |
| all | All in-sync replicas wrote it | Slowest; the safest (with min.insync.replicas ≥ 2) |
No duplicates from retries: the idempotent producer
A producer retries when it doesn't get an acknowledgement, but the first attempt may actually have succeeded, so a naive retry writes a duplicate. With enable.idempotence=true (the default since Kafka 3.0) the producer gets a producer id and numbers each batch per partition. The broker remembers the last sequence per producer and partition, and discards duplicates and rejects gaps. Retries become safe and order is preserved, even with up to 5 requests in flight.
That's why EventPublisher's comment says sends are "never retried by hand: the idempotent producer already retries safely". A send that still fails after delivery.timeout.ms is counted in the manifest (failed), so the integrity check can account for it. The measured run had 0 failed sends.
It deduplicates retries within one producer's lifetime. If the application itself crashes and re-sends after restart, that's a new producer id and a new record. End-to-end duplicates need transactions (section 8) or idempotent consumers.
5. Brokers, replication and KRaft
- Replication factor (RF): each partition has a leader replica (serving reads and writes) and RF−1 followers that copy it.
- ISR (in-sync replicas): replicas caught up with the leader. With
acks=allandmin.insync.replicas=2on RF=3, a write is acknowledged only when at least 2 replicas have it, so one broker can die without losing acknowledged data. - If a leader dies, a controller elects a new leader from the ISR.
Telemetry runs one broker with replication factor 1 (TOPIC_RF=1), fine on a laptop but with no tolerance for a broker failure. The failure you tested was a consumer instance being killed, not a broker. Telemetry's README lists "a second broker with replication factor 3" as next work. Say this plainly if asked about durability.
KRaft: no more ZooKeeper
Older Kafka kept cluster metadata (topics, partitions, leaders) in a separate ZooKeeper cluster. KRaft moves that into Kafka itself: a quorum of controller nodes agrees on metadata using the Raft consensus protocol. Kafka 4.0 removed ZooKeeper support entirely. Your Compose broker runs in combined mode, KAFKA_PROCESS_ROLES: broker,controller, with KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093. Telemetry's notes forbid any ZooKeeper setting in source.
Listeners: the classic Docker confusion
KAFKA_LISTENERS: INTERNAL://:19092,EXTERNAL://:9092,CONTROLLER://:9093
KAFKA_ADVERTISED_LISTENERS: INTERNAL://kafka:19092,EXTERNAL://localhost:9092
A client first connects to a bootstrap address, and the broker then tells it which address to use (the advertised listener). Containers on the Compose network must be told kafka:19092; your laptop's IntelliJ must be told localhost:9092. One listener per audience. That's why the apps use KAFKA_BOOTSTRAP=kafka:19092 in Compose and localhost:9092 by default outside it. Getting this wrong ("connects, then hangs") is the most common Kafka-in-Docker bug.
6. Consumers, groups, offsets and rebalancing
Consumers pull records in a loop (poll()). Consumers with the same group.id form a consumer group: Kafka assigns each partition to exactly one member of the group, so the group divides the work. A different group gets its own full copy of the stream.
suspicious-players (6 partitions) group "judge-agent" (one Judge app, concurrency 3) P0 P1 ─────────────────────────────────► consumer thread 1 P2 P3 ─────────────────────────────────► consumer thread 2 P4 P5 ─────────────────────────────────► consumer thread 3 the verifier reads the same topic in its own group: a full independent copy
Offsets and commits
A group stores its progress, the next offset to read for each partition, in the internal topic __consumer_offsets. On restart or reassignment, consumption resumes from the committed offset. auto.offset.reset decides where a group with no committed offset starts: earliest (from the beginning, which both your consumers use, so nothing published before startup is missed) or latest.
Delivery semantics come from when you commit
| Semantics | Order of work | On a crash |
|---|---|---|
| At-most-once | Commit, then process | The record is skipped (lost) |
| At-least-once | Process, then commit | The record is processed again (duplicate), so make processing idempotent |
| Exactly-once | Process and commit atomically in one Kafka transaction | Only for Kafka-to-Kafka pipelines (module 10) |
The Judge and the Arena bridge are at-least-once with idempotent effects. The bridge relies on Arena's externalId (a replay gets 200 and changes nothing). The Judge checks processed.hasFlag(flagId) against its replayed verdicts, re-reads player status in pre-checks, and handles Arena's 409.
Liveness: heartbeats, sessions and the poll interval
- Consumers send heartbeats. If none arrives within
session.timeout.ms, the member is declared dead and its partitions are reassigned. Telemetry's detectors use 6,000 ms (the broker's minimum) instead of the default 45 s, to notice a killed instance quickly. - Separately, if the application doesn't call
poll()again withinmax.poll.interval.ms(default 5 minutes), the consumer is considered stuck and leaves the group. The Judge processes one flag per poll (max-poll-records: 1), and a flag can take two model investigations plus retries, so it raises the limit to 10 minutes (600000). This is a common production bug: slow processing triggers endless rebalances.
Rebalancing
When members join, leave or die, the group rebalances: partitions are redistributed. Classic "eager" rebalancing stops everyone's consumption during the shuffle; cooperative incremental rebalancing moves only the partitions that need to move. Kafka 4 also brings a new consumer group protocol in which the broker coordinates assignments. Rebalances are a big part of failover time, which module 10 measures (three back-to-back rebalances, 14.4 s).
Consumer lag
Lag = the partition's latest offset minus the group's committed offset: how far behind the consumer is. It's the key health metric of any streaming system. During Telemetry's failover, the per-partition maximum lag peaked near 380,000 events and took about 2 minutes to drain. Client-side lag metrics (records-lag-max) are windowed maxima and lag behind reality; the authoritative view is kafka-consumer-groups.sh --describe, or Kafka UI on port 8090.
7. Retention vs compaction: logs and tables
cleanup.policy | Keeps | Think of it as | Your topics |
|---|---|---|---|
delete | Everything newer than retention.ms (or under a size limit) | An event history | match-events (24 h), flags, DLTs (7 d) |
compact | At least the latest record per key | A table / changelog / snapshot | player-profiles, integrity-stats, cheater-ground-truth, simulation-manifest, judge-verdicts |
- A record with a
nullvalue is a tombstone: it tells compaction to delete that key entirely. Telemetry'sTopicSnapshotReaderhandles tombstones, and an integration test covers them. - Compaction runs in the background and never touches the active segment, so a reader may still see several values for one key. Any code that rebuilds state from a compacted topic must keep the last value per key.
The Judge has no database. Its state is the compacted judge-verdicts topic (latest verdict per flag id). At startup, ProcessedFlagsLoader reads the whole topic into memory before starting the flag listener (autoStartup = "false", then registry.getListenerContainer("judge").start()). The listener can then skip flags it already judged, and apply the "judged in the last 30 minutes" pre-check. The topic is both the output and the memory. This is a light form of event sourcing: state rebuilt by replaying a log.
8. Transactions and read_committed
A transactional producer (configured with a transactional.id) can write to several partitions and commit consumer offsets as one atomic unit:
producer.beginTransaction();
producer.send(record to suspicious-players P2);
producer.send(record to a state-store changelog P2);
producer.sendOffsetsToTransaction(offsets of match-events P2, groupMetadata);
producer.commitTransaction(); // all visible, or (abort) none
This read-process-write cycle is how Kafka Streams achieves exactly-once (module 10). Under the hood, the broker writes special commit or abort markers into each partition.
But aborted records are still physically in the log. A consumer with the default isolation.level=read_uncommitted reads them anyway. With read_committed, the consumer only reads up to the last stable offset and filters out records from aborted transactions. That's why every consumer of detector output (the verifier and the Judge, plus Telemetry's own bridge) sets isolation-level: read_committed. Without it, a detector killed mid-transaction could leak flags that were then rolled back and re-emitted, which would look like duplicates.
Kafka transactions cover Kafka writes and Kafka offsets only. The Judge's HTTP call to Arena can't take part. That's why the Judge is at-least-once with idempotent, state-checked effects, and why the bridge leans on externalId. Saying "exactly-once inside Kafka; at-least-once plus idempotency at the boundaries" shows you really understand it.
9. Creating topics deliberately
@Bean
NewTopic matchEventsTopic() {
return TopicBuilder.name(Topics.MATCH_EVENTS)
.partitions(6)
.replicas(replicationFactor)
.config(TopicConfig.CLEANUP_POLICY_CONFIG, TopicConfig.CLEANUP_POLICY_DELETE)
.config(TopicConfig.RETENTION_MS_CONFIG, "86400000") // 24 h
.build();
}
Spring's KafkaAdmin creates any missing NewTopic beans at startup. The broker has KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false", so a typo in a topic name fails loudly instead of silently creating a new topic with default settings. The Judge declares NewTopics only for the topics it owns (judge-verdicts and its DLT) and never creates or changes Telemetry's: ownership again.
10. Message formats and schema evolution
All values are JSON produced by your own Jackson 3 serde. JSON is readable and flexible, but nothing enforces the shape: a producer can rename a field and every consumer breaks, or silently loses data (the skillTier curveball in module 01).
- Avro or Protobuf with a Schema Registry: producers register a schema; the registry rejects incompatible changes; records carry a schema id and are compact binary.
- Compatibility modes: backward (new consumers can read old data: add optional fields only), forward (old consumers can read new data), full (both).
- Telemetry's next steps include "Avro with Schema Registry for
MatchEvent, with a demonstrated backward-compatible change".
11. Spring Kafka
Producing with KafkaTemplate
template.send(topic, key, value) returns a CompletableFuture<SendResult> that completes when the broker acknowledges. The simulator collects thousands of these and waits with allOf(...).join(). The Judge's listener calls publisher.publish(verdict).join() so it doesn't return (and let the offset be committed) until the verdict is safely stored.
Consuming with @KafkaListener
@KafkaListener(id = LISTENER_ID, topics = "${judge.kafka.flags-topic}", autoStartup = "false")
void onFlag(ConsumerRecord<String, String> record) {
SuspiciousPlayer flag = parseFlag(record.value()); // bad JSON → NonRetryableJudgeException
if (processed.hasFlag(flag.flagId())) {
metrics.duplicateFlag();
return; // already judged: idempotent
}
JudgeVerdictRecord verdict = judgeService.handle(flag); // pre-checks, models, Arena action
publisher.publish(verdict).join(); // wait for the broker's ack
processed.remember(verdict);
}
- Spring runs a listener container that polls and calls your method for each record.
concurrency: 3creates three consumers in the group (at most 6 useful, one per partition). ack-mode: record(Judge): commit the offset after each successfully processed record, rather than per poll batch.- Gotcha from Telemetry: by default a listener's
idis also used as its group id, overriding thegroup-idin YAML. The bridge setsidIsGroup = falseso it keepstelemetry-arena-bridge. autoStartupcan be a property: the bridge uses"${telemetry.arena.enabled:false}", so it only runs when Arena integration is switched on.
Error handling: retries, back-off and dead-letter topics
When the listener throws, Spring's DefaultErrorHandler decides what to do. It doesn't commit the failed record; it seeks back so the record is delivered again after a back-off. After the retries are exhausted, a recoverer runs, typically the DeadLetterPublishingRecoverer, which publishes the record (with exception details in headers) to a dead-letter topic (DLT) and lets consumption move on.
ExponentialBackOffWithMaxRetries backOff =
new ExponentialBackOffWithMaxRetries(3);
backOff.setInitialInterval(1000); // 1 s
backOff.setMultiplier(2.0); // 2 s, 4 s
DefaultErrorHandler handler =
new DefaultErrorHandler(countingDltRecoverer, backOff);
handler.addNotRetryableExceptions(
NonRetryableArenaException.class);
Temporary Arena failures (429, 5xx, timeouts): 4 attempts, then match-summaries.DLT. Permanent ones (400/404/409/422): straight to the DLT.
DefaultErrorHandler handler = new DefaultErrorHandler(
recoverer,
new FixedBackOff(5000, // every 5 s
FixedBackOff.UNLIMITED_ATTEMPTS)); // forever
handler.addNotRetryableExceptions(
NonRetryableJudgeException.class);
Outages (Claude or Arena down, circuit open): hold the flag and retry forever, never dropping it. Malformed flags and Claude 400s: straight to suspicious-players.judge.DLT.
Classify failures. A transient failure (the other side is down or busy) deserves retries, because waiting can fix it. A permanent failure (the message itself is bad: a poison pill) will fail forever, so retrying just blocks the partition. Send it to a DLT for a human to inspect. The two projects choose different retry budgets for good reasons: a match summary is low-value and the bridge gives up after about 7 seconds; a flag is a possible cheater, so the Judge never gives up during an outage. ResilienceIT proves it: during a 20-second injected outage no flag was dead-lettered, and all 20 were judged afterwards.
While a record is being retried, the rest of its partition waits behind it. That's correct when order matters and the cause is a shared outage (everything would fail anyway). When failures are per-record, Spring Kafka's non-blocking retries (@RetryableTopic, with retry topics and delays) let the partition keep flowing, at the cost of ordering. Another subtle point: the DLT recoverer writes to the same partition number as the source record, so the DLT needs at least as many partitions, which is why they match.
12. Kafka vs alternatives
| Need | Good fit |
|---|---|
| High-throughput event streams, replay, many independent consumers, stream processing | Kafka |
| Task queues, per-message acknowledgement, routing, priorities, delayed messages | RabbitMQ, SQS |
| Small scale streams alongside an existing Redis | Redis Streams |
| Managed, serverless pub/sub | Google Pub/Sub, AWS Kinesis, SNS |
13. Interview questions
What is a partition and why does Kafka have them? Basic
A partition is an ordered, append-only log; a topic is a set of them. Partitions give parallelism (different brokers store them, different consumers read them) and are the unit of ordering: order is guaranteed only within a partition. I key every player event by player id, so one player's events land in one partition, in order, which per-player windows and sequence checks require.
How do consumer groups work? Basic
Consumers sharing a group.id split the partitions: each partition goes to exactly one member, so the group processes the topic once, in parallel. Another group gets its own complete copy with its own offsets. Parallelism is capped by the partition count. When members join or leave, the group rebalances.
What does acks=all plus idempotence give you? Intermediate
acks=all means the leader acknowledges only once all in-sync replicas have the record, so an acknowledged write survives a leader failure (given min.insync.replicas ≥ 2 and RF ≥ 3). Idempotence gives each producer an id and a sequence number per partition so the broker discards duplicate retries and keeps order. Together: no loss of acknowledged data and no duplicates from producer retries. In my setup RF is 1 on a single broker, so I'm honest that broker loss isn't covered.
At-least-once vs exactly-once: which did you use where? Intermediate
Inside Telemetry's detector, Kafka Streams with exactly_once_v2: reads, state updates, outputs and offset commits are one Kafka transaction, which I verified per player with sequence numbers. At the edges, the bridge to Arena and the Judge, it's at-least-once with idempotent effects, because HTTP calls can't join a Kafka transaction: Arena dedups matches by externalId, and the Judge skips flags already in its verdict log and re-checks player status.
What does isolation.level=read_committed do and why do your consumers need it? Deep
Transactional producers write records before committing; on abort the records stay in the log with an abort marker. A read_uncommitted consumer would read them. read_committed reads only up to the last stable offset and skips aborted records. The detector writes inside transactions, and if an instance is killed mid-transaction, its partial output is aborted and the new owner re-produces it. Without read_committed, the Judge and the verifier would see those aborted flags plus the re-emitted ones, which look like duplicates.
Retention vs compaction? Intermediate
Delete retention keeps records for a time or size and then drops whole old segments: a history. Compaction keeps at least the latest value per key and removes older ones in the background: a table. I use compaction for profiles, integrity stats, the manifest, ground truth and the Judge's verdicts, which it replays at startup as its only state. Tombstones (null values) delete keys, and because compaction lags, a replay must keep the last value per key.
How do you handle a poison message? Intermediate
Classify the exception. A poison message fails deterministically, so retrying blocks its partition forever. Spring Kafka's DefaultErrorHandler with addNotRetryableExceptions sends those straight to a dead-letter topic via DeadLetterPublishingRecoverer, with exception headers for diagnosis, and consumption continues. Transient failures get back-off retries. In Telemetry's detector I go further: raw bytes are parsed inside the topology, so a corrupt event becomes a dead-letter record with its offset and Base64 payload instead of a deserialization exception.
Why does the Judge retry forever while the bridge gives up after 3 retries? Deep
Different values and different failure causes. A flag is a possible cheater: dropping it is costly, and the failures that block it (Claude or Arena unavailable, circuit open) are outages that end, so the Judge holds the flag and retries every 5 seconds without committing the offset. Only a malformed flag or a Claude 400 goes to the DLT. A match summary that keeps failing after about 7 seconds is either a real outage or a permanent rejection; parking it in the DLT keeps the partition moving and it can be replayed. Same mechanism, different budgets.
Why set max.poll.interval.ms to 10 minutes in the Judge? Deep
If the app doesn't call poll() within that interval, Kafka assumes it's stuck, removes it from the group and rebalances, and the uncommitted record goes to another member, which may also time out. Judging one flag can involve two model investigations with tool calls, plus resilience retries, so the default 5 minutes could be exceeded. I take one record per poll and allow 10 minutes. The alternative is to pause partitions and process asynchronously.
What is consumer lag and how did you watch it? Intermediate
Lag is the log-end offset minus the group's committed offset: how many records are waiting. I graphed it in Grafana and checked it with kafka-consumer-groups.sh --describe, which is authoritative. Client metrics like records-lag-max are windowed maxima and trail reality. During the chaos test the per-partition max peaked around 380,000 and took about 2 minutes to drain, because two survivors had only about 5,000 events/s of headroom over the 25,000/s input.
Your app connects to Kafka in Docker and then hangs. What do you check? Curveball
Advertised listeners. The bootstrap connection works, but the broker then hands back its advertised address, for example kafka:19092, which the host can't resolve, or localhost:9092, which inside a container points to the container itself. Configure one listener per audience: INTERNAL://kafka:19092 for containers, EXTERNAL://localhost:9092 for the host, as Telemetry's Compose file does.
Can you just add partitions to match-events later to scale? Curveball
You can, but it remaps keys: hash mod 6 differs from hash mod 12, so a player's new events land in a different partition from their old ones. Per-key ordering breaks around the change, and stateful processing (windows keyed by player) and co-partitioned joins break, because player-profiles would still have 6. You'd plan capacity up front, or create a new topic with more partitions and migrate.
What did KRaft change? Basic
Kafka no longer needs ZooKeeper. A quorum of controller nodes stores cluster metadata in an internal log and agrees using Raft. It's one system to run, with faster controller failover and better metadata scalability. Kafka 4 is KRaft-only; my broker runs as a combined broker and controller.
14. Quick quiz
1. Kafka guarantees order:
Key by entity to keep each entity's events in one partition.
2. A group has 8 consumers for a 6-partition topic. How many are idle?
One partition per consumer at most.
3. Processing first and committing the offset afterwards gives:
A crash in between means reprocessing, so make effects idempotent.
4. A compacted topic keeps:
It behaves like a table, and is great for state.
5. Without read_committed, a consumer of transactional output may see:
Aborted records remain in the log; read_committed filters them.
6. Which failure should go straight to a DLT without retries?
Deterministic failures are poison pills.
7. The idempotent producer prevents:
Per-session sequence numbers let the broker drop retried batches.
15. Cheat sheet
- Kafka = a distributed, partitioned, replicated append-only log; readers keep offsets; replay is possible.
- Partitions: parallelism + per-partition order; key →
murmur2 mod n; max consumers = partitions; adding partitions remaps keys. - Producer: batching (
batch.size,linger.ms), lz4,acks=all, idempotence (producer id + sequence: no retry duplicates). - Replication: leader + followers, ISR,
min.insync.replicas. Yours: 1 broker, RF 1 (say it). - KRaft: Raft controller quorum, no ZooKeeper (Kafka 4). Advertised listeners: INTERNAL
kafka:19092, EXTERNALlocalhost:9092. - Groups: a partition per member; offsets in
__consumer_offsets;earliest; heartbeats +session.timeout.ms(6 s in the detectors);max.poll.interval.ms(10 min in the Judge); rebalances; lag. - Semantics: commit-then-process = at-most-once; process-then-commit = at-least-once (+ idempotency); a transaction = exactly-once (Kafka only).
- Retention (history) vs compaction (latest per key; tombstones; keep last on replay).
judge-verdicts= the Judge's state, replayed before the listener starts. - Transactions: atomic multi-partition writes + offsets; consumers need
read_committed. - Spring Kafka:
KafkaTemplate(future),@KafkaListener(concurrency,idIsGroup=false,autoStartup),DefaultErrorHandler+ BackOff + not-retryable +DeadLetterPublishingRecoverer(same partition). - Retry budgets: bridge 1/2/4 s then DLT; Judge every 5 s forever on outages, DLT only for poison.
- Next: Avro + Schema Registry; RF 3 cluster.