Back to the channels

#java

11:57 Para https://bytecode.news/posts/2026/03/kafka-offset-commit-with-spring-boot
11:58 dreamreal Para: ++
11:58 nevet Para now has karma of 1.
11:59 dreamreal the kafka commit semantics are ... interesting, and how spring boot interacts with them can be unexpected
11:59 dreamreal it treats the *read* as a transaction
11:59 jreicher Why wouldn't it?
12:00 jreicher It probably shouldn't be called a "read". It's a really a consume.
12:00 dreamreal Well, it doesn't consume the content, it IS a read
12:00 jreicher It does consume the content. The commit offset is committed back, so you don't re-consume the same message by mistake.
12:00 dreamreal but treating the read as a transaction can have odd semantics because the transaction doesn't necessarily propagate unless you take steps
12:01 dreamreal it only consumes the content from that client-id's perspective, though
12:01 dreamreal so I'd say it's still a read
12:01 jreicher I'm happy to use the word read, but the offset is how the "consume effect" is implemented.
12:02 dreamreal terminology is a drag, yeah, I get that
12:02 jreicher So because there's a mutation taking place, it needs to be a transaction.
12:03 Para Kafka is built to synchronize on write acks, so that's one factor to consider as well. For the most part it doesn't matter much.
12:04 Para One wild choice was using Zookeeper ephmeral nodes for consumer group topic offset tracking, although I think they got rid of that, as there's no zk dependency anymore.
12:04 Para (or something similar, my brains a bit extra mushy right now)
12:05 jreicher I never took the time to understand zookeeper, but yeah I think it's gone.
12:06 Para graphy thing with awesome features and nanosecond-scale latency as long as the hardware doesn't suck or behave erradically.
12:06 dreamreal zk isn't gone but you can deploy without it, I think
12:06 Para It never survived the cloud that well, CPU steal basically was like a gutpunch to ZK and then it would get into split brain dance, among other issues :D
12:07 Para Kafka was built in Linkedin's in-house data centers originally with all the hardware they wanted running on bare metal, so they obviously hadn't considered things like CPU steal initially.
12:08 jreicher Whenever I see split brain I have visions of some developer somewhere thinking how easy it was to write that automatic failover function...
12:08 Para As a thingy I do like the ideas of Zookeeper though. Hierarchical configuration management with clustering, synchronization, several persistence levels etc.
12:08 Para Kinda like what Hashicorp's Consul would then do a bit later.
12:55 deebo never knew kafka was from linkedin
12:57 Para Yeh, pre-Microsoft
12:57 Para Then forked to Confluent.
12:57 dreamreal life is a highway, man
12:57 dreamreal signs everywhere
12:58 deebo tom cochrane time
12:58 dreamreal It's a great song! And it's incredibly fun to play.