deebofor some reason assumed yourkit snapshots would include jmx data, but no
deeboany good options for recording and graphing micrometer data locally?, hopefully without running an open/elasticsearch cluster + grafana etc
ParaI run Jaeger tracing through container :D
ParaI'd wager depends on what you want out from the system.
ParaI also always print trace/span id:s to each log row, general ideology being that as I have these systems, I've made them always available and linked across everything to let myself be lazy.
ParaEven the REST API endpoints return those id:s so I can use browser dev tools to spy on what's going on.
ParaWhat else...IDEA has actuator based live stats for service, that works as well.
javabotPara's title: "Tutorial: Explore Spring support features | IntelliJ IDEA Documentation"
ParaWhat I'm trying to say - it depends :)
deeboi'm starting yourkit agent on startup in a docker swarm environment, doing a rolling upgrade to simulate a deployment and then after a few minutes i bring the system down and the agent writes snapshots to disk
deebobut yourkit snapshots can't be filtered by time, and i can't correlate e.g. jdbc onnection acquisition times
deeboat this point i'd need more of an apm than cpu profiling
deebomaybe i'll jhust have to deploy to our real env to get a real apm agents stats, just slows the roundtrips for testing by 10x, really annoying
deeboor i have to programmatically start the yourkit profiling so i get exactly what i want in the snapshot
deebodidn't notice until now but yourkit agent has a periodic snapshot feature, so could capture 1 minute slices to find a good set of data, but i added a hook on startup (when ready for traffic) to profile for 90 seconds, save snapshot and stop profiling
deebohope this gives me something useful
dmlloydmaybe you want JFR, or maybe look into async profiler
dreamrealSDL has decided to disallow AI use in the project, which I find interesting
dreamrealI understand it, but I think that's a bad response to a real problem
deeboyeah have to check jfr at some point
deebobut i think i finally found the issue, hikaricp creates new connections sequentally blocking, so if you have a buttload of traffic suddenly coming in, everything takes ages, but there's a switch to have it block init until minimum-idle connections are created
dreamrealHow long is it taking to open a new connection?
deebohave to measure at some point, but on a 1cpu 2gb node that does 99.95% io bound work, all that traffic and initializations seem to be slowing it down, and is the onyl consistent blip in profiling
deebo99% response times go from 20ms to 1000ms for ~15sec
deeboduring a rolling node-by-node deploy
dreamrealto *connect to a database*? Yikes.
deeboyourkits tracing has way too much overhead, and in sampling mode it gets really confused by virtual threads
dreamrealThe guy that's about used JFR and JMC for virtual threads
deeboyeah have to test jfr, and complain to yourkit
deebojfr is hella confusing compared to yourkit, but have to look deeper into the method timing/profiling stuff, but execution sampling is useless in an io bound app, most cpu used was a ConcurrentHashMap at 0.8% that something uses for caching