* CodeGeek!~codegeek@about/java/CodeGeek changed the topic to: Welcome! || Read Channel Rules at https://javachannel.org/ before participating. || Paste limit is two lines; ~pastebin lists options. || No applets, please. || Minecraft, Android, and Javascript all have their own channel. || You are being logged.
dreamrealthe kafka commit semantics are ... interesting, and how spring boot interacts with them can be unexpected
dreamrealit treats the *read* as a transaction
jreicherWhy wouldn't it?
jreicherIt probably shouldn't be called a "read". It's a really a consume.
dreamrealWell, it doesn't consume the content, it IS a read
jreicherIt does consume the content. The commit offset is committed back, so you don't re-consume the same message by mistake.
dreamrealbut treating the read as a transaction can have odd semantics because the transaction doesn't necessarily propagate unless you take steps
dreamrealit only consumes the content from that client-id's perspective, though
dreamrealso I'd say it's still a read
jreicherI'm happy to use the word read, but the offset is how the "consume effect" is implemented.
dreamrealterminology is a drag, yeah, I get that
jreicherSo because there's a mutation taking place, it needs to be a transaction.
ParaKafka is built to synchronize on write acks, so that's one factor to consider as well. For the most part it doesn't matter much.
ParaOne wild choice was using Zookeeper ephmeral nodes for consumer group topic offset tracking, although I think they got rid of that, as there's no zk dependency anymore.
Para(or something similar, my brains a bit extra mushy right now)
jreicherI never took the time to understand zookeeper, but yeah I think it's gone.
Paragraphy thing with awesome features and nanosecond-scale latency as long as the hardware doesn't suck or behave erradically.
dreamrealzk isn't gone but you can deploy without it, I think
ParaIt never survived the cloud that well, CPU steal basically was like a gutpunch to ZK and then it would get into split brain dance, among other issues :D
ParaKafka was built in Linkedin's in-house data centers originally with all the hardware they wanted running on bare metal, so they obviously hadn't considered things like CPU steal initially.
jreicherWhenever I see split brain I have visions of some developer somewhere thinking how easy it was to write that automatic failover function...
ParaAs a thingy I do like the ideas of Zookeeper though. Hierarchical configuration management with clustering, synchronization, several persistence levels etc.
ParaKinda like what Hashicorp's Consul would then do a bit later.
* michele joined #java
* akaWolf joined #java
* polarian joined #java
deebonever knew kafka was from linkedin
ParaYeh, pre-Microsoft
ParaThen forked to Confluent.
dreamreallife is a highway, man
dreamrealsigns everywhere
deebotom cochrane time
dreamrealIt's a great song! And it's incredibly fun to play.