Back to the channels

#java

08:06 Bombe I don’t think that even compiles, which makes it doubly worthless!
08:09 jreicher It's art, apparently. Heathen.
08:10 Bombe Can’t argue with that, I guess.
10:07 enoq so we're gonna build a backend for frontend application pretty soon; looking at virtual threads, we could use spring servlet APIs and get a lot of performance out for free, but I'm unsure if that's a good idea if most of your application will do I/O and potentially wants to do API requests in parallel
10:08 enoq from what I've seen structured concurrency APIs are in preview and I don't really know how easy/well those would integrate into a classic spring boot servlet server; do I want to go with reactive instead?
10:09 Para Use the cache, Luke :)
10:10 enoq one issue I could see when going reactive is probably hibernate; there's a reactive version available, but I'm unsure how well those work; I'll definitely go for stateless sessions btw
10:11 Para FWIW I've found Guava's caches really useful in that. Create a cache loader which has the heavy operation inside it and let requests wait/park until result is fetched by the first caller and value result is cached.
10:12 Para And definitely do things cache first, adding them as an afterthought is not fun.
10:12 ernimril enoq, how many concurrent users are you planning to have? how long time does each request take? Do you have to care about thread handling?
10:13 enoq ernimril, I don't have data yet for concurrent users; customer is very performance sensitive and has set ms response time goals
10:13 Para Then remove Hibernate entirely.
10:13 enoq technically you could ofc also spin up multiple servlet apps
10:14 ernimril "ms response time goals"? as in 999 ms or as in 1 ms?
10:14 enoq below 100 IIRC
10:14 Para You'll also need proper tracing to detect hotspots, design a multi-layered cache as well. Avoid distributed caches such as memcached, their worst case scenarios/faults will kill you.
10:14 ernimril enoq, on average or worst case?
10:15 Para Or is that 99%?
10:15 enoq ernimril, great question, I think it's probably an average but would have to look into it
10:15 ernimril these things matter a lot. if it is average you can do almost anything a 100 ms is a _long_ time for a computer... :-)
10:16 Para On tracing, you need to be able to prove _your_ part. If the request takes 60ms to come from client to your backend and that is counted as part of it, you definitely want to be able to say that someone else ate your quota... :)
10:16 ernimril now, if you connect to a database over a filtering tunnel and going through another country you may still have problems (yes, this has happened to me)
10:16 enoq right, the thing that sorta gives me headaches is that we potentially have a nextjs server app in front as well
10:17 enoq we're gonna host in the same data center though (Azure) so I hope latencies will be low
10:17 Para I did as an exercise an opentelemetry tracing Spring aspect some time ago, I could share that if you're interested. And I can find it, it's in my pile of experiments :P
10:18 enoq sure
10:18 enoq the reason why I'm asking is because I think the need for webflux might go away once we've got structured concurrency in java
10:19 jreicher enoq: I'm probably saying something Para has already said a different way, but make sure you solve observability for your performance metrics if that's something you care about. Realistic workloads are difficult to reproduce so if you see something weird happening, that'll probably be your only chance to know why it happened.
10:19 ernimril enoq, with the loose requirements you have today I would probably just go with whatever seems most easy (spring boot if you know and use that already, but perhaps quarkus)
10:20 enoq classic case of premature optimization?
10:20 ernimril measure, then fix, but try to plan well so you get good algorithmic complexity
10:33 jreicher Measuring is really hard for stuff like this, which is why the method of measurement needs to be planned and instrumented.
11:19 deebo enoq: we use spring boot 3 and 4 exclusively with virtual threads in a microservice architecture where everything is just waiting for io most of the time
11:19 enoq I mean yeah, virtual threads work really well
11:19 deebo as long as you're on jdk24+ to avoid pinning issues, there's really no need to do rx and get cancer
11:19 enoq just more concernced about parallel requests
11:19 enoq also looking into Kotlin coroutines btw
11:20 deebo does't really matter, we just mostly do CompletableFuture.supplyAsync(() -> api.call(), virtualThreadsExecutor), but on spring @async would work just as fine
11:21 deebo one service is graphql which does one root query, then massive amounts of parallel queries to fill the graph
11:22 deebo we don't really even do caches, except for apis that were not designed to be used like this and for example return all data at once without any filtering
11:28 dreamreal enoq: virtual threads in spring are very good
11:28 dreamreal what problems are you actually having
11:29 enoq dreamreal, I don't want reactive but I want to make use of parallel http requests/db calls
11:29 enoq my impression was that I need to wait for structured concurrency in the jdk first to get that
11:30 enoq but looks like spring has @Async
11:30 dreamreal you don't need any of it
11:30 dreamreal just write ordinary blocking servlets
11:30 dreamreal then turn on virtual threads
11:30 dreamreal (requires java 21 or later)
11:31 dreamreal You'll get 95%+ or better of the reactive performance just by doing that, and that remaining percentage affects a remarkably small number of apps
11:34 dreamreal and your code complexity will drop like a stone.
11:35 dreamreal Easier to debug, easier to reason about, easier to read, easier to do everything.
11:39 enoq right, I just need an escape hatch if I ever need to optimize that use case
11:40 dreamreal Are you working for netflix, amazon, anyone like that?
11:42 dreamreal Is who you're working for expecting to become netflix, amazon, anyone like that? I mean, I know we all plan for it, but even so
11:43 enoq no, they are using Magento and other PHP based software right now and migrate off because of performance issues, so it's an important topic to be able to handle that
11:44 enoq or at least have good arguments set aside if it comes up
11:45 enoq software architecture itself is a tricky topic to navigate since there's so much dogma floating around
11:46 dreamreal well. Here's more: YAGNI applies. For 99.999% of apps, including every damn one of mine, virtual threads in spring will provide everything you're looking at reactive programming to give you.
11:46 enoq correct, try to argue that against a competitor who throws performant async style programming requests at you ;D
11:46 dreamreal The thread mentions caching, and maybe caching can help, too, but that's to reduce certain aspects of the process, and I don't know what you're doing, so I can't point at caching and say "yes, you want that too."
11:46 enoq + nodejs
11:47 dreamreal enoq: competitor... internally or externally? nodejs' async programming isn't exactly a panacea, plus it's nodejs
11:48 enoq external; exactly, nodejs is way slower, still need a way to quickly play the bullshit bingo
11:48 dreamreal what the heck? Get an SLA, meet it. Your external competitor is going to be dust.
11:49 dreamreal Turn on virtual threads, boom, you have "industry standard scalability" with commodity programmers.
11:50 dreamreal You don't get to use the kickiest, hippest new apis, I guess, and you're not at the mercy of structured concurrency Feature O' Th' Day, and you don't get to claim to be one of the thousands of projects underusing reactive programming, I suppose, but personally I don't consider those weaknesses, I'd rather get stuff done
11:51 enoq fully agree; as mentioned, I just need a back up argument, but @Async looks like it will provide that
11:51 dreamreal Why
11:52 dreamreal spring.threads.virtual.enabled=true # provides that with no source change
11:52 julemand101 enoq: What is your backend doing that would require CPU cycles? In most applications, the slow part is going to be IO and especially database queries so if the same is the case for you, then changing to Java would not really give much benefits.
11:53 deebo in my experience you get the most out of it as long as you have http and db connection pooling
11:53 enoq julemand101, I don't except any number crunching; almost everything will be I/O, be it REST calls or db queries
11:53 dreamreal enoq: then smart caching of consistent resources is going to be your next lever
11:54 dreamreal this is not rocket surgery, it's barely brain salad
11:54 deebo we could run our 5+ country multitenant systems main data service on 1vcpu and 2GB mem since it's mostly io with no actual cpu work
11:55 deebo but for high availability we have "excess" capacity that's just idle :) virtual threads are basically magic in these types of services
11:55 julemand101 enoq: Ok, so it is really the best way to spend your time/money to rewrite your backend into a different language? I mean, it sounds like you don't have a good understanding of where your performance bottlenecks are if you suspect your PHP code when your application are mostly IO bound?
11:55 enoq julemand101, no rewrite happening
11:55 julemand101 Ah so there are no backend at the moment?
11:56 enoq customer wants to transition to a cloud provider which the current backend does not support
11:56 dreamreal okay, now I'm concerned.
11:56 Drixtan enoq: there are 2 ways of handling the blocking IO problem: green threads and the "async" model (which is a stateful machine behind the scene). The "async" model, which has been popularized by C#, is known to "color" your methods with the "async" keyword (see typescript, rust, C#...). This put the code in a situation that you often have to duplicate the code: one async, one sync - where the "function coloring". The second method to manage the blocking IO is li
11:56 Drixtan ke Java virtual threads or Golang green-threads for example. This approach does not color your code, so it eliminate duplication of code.
11:56 dreamreal enoq: ... you need to define scope and roles here
11:56 Drixtan enoq: as per webflux vs virtual threads, webflux was created before the virtual threads to manage this IO blocking problem, so enjoy the new pattern and use the virtual threads.
11:56 enoq I think we're going off on a huge tangent here based off a misunderstanding
11:56 dreamreal enoq: relying on #java for architecture is ... not the worst advice, I guess, in that we CAN know what we're talking about, but it's certainly still terrible advice
11:57 dreamreal enoq: yes, agreed.
11:58 dreamreal enoq: everything you've said looks TO ME like you have a spring boot app to write, and you want it to scale horizontally as well as vertically, which is pretty easy - and the easiest performance spice in spring is to write simple code with simple services (a traditional architecture) with virtual threads turned on.
11:58 dreamreal BUT I have NO insight into the app or the SLA.
12:00 dreamreal that *approach*, plus caching resources that are expensive to retrieve but are generally constant otherwise, catches *a ton* of response time issues. But I don't know the problem space being talked about, so ANY advice has to be taken with a lot of salt and an eye to how much you've paid for it.
12:00 dreamreal the only person with skin in the game here is YOU.
12:00 enoq right, I fully agree with that
12:01 Drixtan I think the real question was: are you safe going with Java? I think you get the answer: yes you are - performance won't be an issue, you always have a good way to optimize, you are not painting yourself in a corner.
12:03 dreamreal enoq: @Async does not help - it creates an analog to a pattern you DO NOT NEED. You can use it, but doing so is mostly doing node-like things in Java, as opposed to java-like things in java. Do java-like things in java.
12:04 enoq how do you handle slow parallel requests to different REST endpoints then?
12:04 dreamreal depends on what "slow" means! Virtual thread dispatch means that each blocking call blocks *only its virtual thread*
12:05 dreamreal like, okay, thread-1927 is taking 4ms to run this blocking call, oh no, while thread-1938 executes the same blocking call in ANOTHER thread, where's the competition for scarce resources?
12:05 enoq let's imagine you need up to date data from endpoint a and b and both take 100ms to complete; if you write it naively using virtual threads, you'll end up with 200ms
12:05 dreamreal node and python treat *threads* as scarce resources, so async makes sense
12:06 dreamreal enoq: why? Why are they executing sequentially?
12:06 enoq that's my question: what's the recommended way to parallelize this using virtual threads
12:06 dreamreal turn on virtual threads.
12:06 dreamreal That's the recommended way to parallelize that.
12:07 dreamreal that's all.
12:07 dreamreal ... that is, in spring.
12:07 Para Yeah, it took over a decade to materialize as a feature because they wanted it to be a turnkey thing.
12:07 Para Flip the switch and move on :)
12:07 enoq the 100ms requests are launched inside spring, not the frontend
12:07 dreamreal enoq: right.
12:08 dreamreal enoq: dude, I've done this. For a long time.
12:08 enoq so just turning on virtual threads will not solve this, you need to fork off threads or similar stuff
12:08 dreamreal why?
12:08 dreamreal Why do YOU need to fork off threads, if the framework does it for you?
12:09 enoq oh, I think I might have misunderstood how they work
12:09 Drixtan enoq: spring 3.2+ the request handler is a virtual thread, imagine you have an "infinite" amount of "threads"
12:09 dreamreal I mean, with virtual threads turned on in spring, requests a and b will both spin up a thread without you doing anything, and they'll both run parallel; is the "up to date data" the blocking point? Can IT handle only one request at a time?
12:09 enoq is it intelligent enough to realize that both method calls have no dependency on each other and can execute in parallel?
12:10 dreamreal yes, in fact you have to work to get it to NOT run in parallel
12:10 Para ...lock pool
12:10 dreamreal we've been TRYING to give you good advice here
12:10 enoq I see, I was under the impression that it suspended execution immediately once it hit the first I/O call
12:10 dreamreal oh god no
12:10 dreamreal that'd be bonkers mad
12:11 Para I/O is not magical, despite especially functional programming folks wanting to make it seem like it :P
12:11 dreamreal that's why I said that you get most of what you want with the single flag
12:11 dreamreal maybe not all of it: if you're netflix or amazon or a very few others, well, you want to manage concurrency differentl
12:12 enoq thank you, that cleared things up massively :)
12:12 enoq and tbh makes it even more impressive
12:12 Para I bet someone's been crazy enough to make lmax disruptor to manage tasks in Spring somewhere at some point but...don't :)
12:12 dreamreal one of my clients was saying "oh nooo what if we have to arrrgh" and I told them that if we could measure against their SLA and we could saturate within 10% of what the actually expected I'd not only eat my hat but I'd buy the hat to eat it AND we'd go reactive. Didn't happen. At all.
12:13 dreamreal Para: that's a bet you'd win, plus EAI is a thing
12:25 deebo seeing one reactive stack trace should put you off the concept entirely
12:26 deebo only annoyance with virtual threads i've run into, is that at least with sampling tracing in yourkit, you don't get proper grouping of samples, due to the way it's actually run on a few platform threads you get weirdly disconnected results
12:30 dreamreal and as far as annoyances go, that one's pretty mild.
12:30 dreamreal at least, in my experience.
12:31 deepy only annoyance I got with virtual threads is that they worked so well that I forgot my app was running on a slow single vCPU in a constrained environment and then it very unexpectedly crashed under load
12:31 jreicher Para: I/O might not be magical, but it doesn't exist within any language.
12:31 jreicher That's not specific to functional programming.
12:31 Para jreicher: FP people are insufferable about I/O at times, is my point :)
12:32 jreicher I think they're just defensive. They don't want to admit they're bad at explaining their preferred paradigm.
12:33 dreamreal or that their holy writ has giant chasms that undercut their magic claims
12:33 dreamreal FP is incredibly useful and I wish they wouldn't keep trying to sell people on it
12:33 dreamreal just back up your shit, that's enough
12:33 jreicher I believe FP is as good as some people say, but there are other people who make false claims about it
12:37 dreamreal FP's great
12:37 deebo i'm just a highly paid data plumber (actual plumbers might make more!), the more functional and immutable stuff is the easier :)
12:37 cheeser if you're strong enough
12:38 dreamreal deebo: sure, like I said, FP's great
12:38 dreamreal it's just not MAGIC
12:38 dreamreal magic's not real, just like unicorns aren't real, wendigos aren't real, birds aren't real
12:39 cheeser finland isn't real
12:39 deebo yep, the full paradigm just doesn't work for me, i've used xmonad (haskell based window manager) for like 15 years or something, and i still have no clue how my configuration works, just glad it does
12:48 jreicher deebo: what "doesn't work" about the paradigm for you? (I have a vested interested in this question)
12:49 Para deebo: Time to descend another level and start using nixos, then.
12:49 Para I'm self-projecting, I have a used laptop arriving soon which I'll be installing nixos onto.
14:09 deebo jreicher: i just can't read it, with haskells custom operators it's even worse
14:44 enoq let's say your app calls a couple of REST APIs that you don't control (but you can add test data); what's your preferred way to test that? spin up a mock rest server? mock the http call? add test data to said REST API?
14:44 Para Mock the call, probably.
14:45 dreamreal for THAT, mock the call, although I personally prefer spinning up a fake server that returns predetermined data
14:45 enoq any reasons why you wouldn't use test data from the live system?
14:46 Para Legal.
14:46 dreamreal ^^^
14:46 dreamreal you don't want to rely on the live system
14:47 enoq thank you
14:52 sbalmos the most advanced company I worked at for a year contractually had a mostly-automated weekly system that took a snapshot of prod on Saturday nights overnight and sanitized as much as it could into normative names etc
20:07 jreicher deebo: interesting. Is it any better when do notation is used?