Feature: Adding basic native memory extraction. No parsed summary yet.

[Delawen]

Very basic native memory statistics output. It still needs some parsing of the raw data to get a better statistics overview.

[Delawen]

How it looks on my machine right now:

$ ./measure-boot.sh
Container 'jdk23-leyden-build' completed bootstrap in 10597 milliseconds
It used the following native memory:
Total: reserved=1984MB, committed=156MB
Container 'fedora-standard' completed bootstrap in 9381 milliseconds
It used the following native memory:
Total: reserved=1930MB, committed=116MB
Container 'jdk23-build' completed bootstrap in 12930 milliseconds
It used the following native memory:
Total: reserved=1931MB, committed=117MB

[franz1981]

Maybe we can use redhat-developer/vscode-java#3395 (comment) or directly extract the same info from ps/pmap -x, too, to save bothering the JVM with additional work which could affect the measurement

[Sanne]

Should we maybe have different sets of JVM parameters depending on what we're going to be measuring?

There's several more configuration flags which I'd like to enable to improve diagnostics (e.g. -Xlog:cds and -Xlog:scc ) but I refrained from enabling them as I fear they would skew our other measurements. So there's also a need for "give me a full report of what's going on, even if it impacts metrics".

Not sure how to best do this, thoughts?

[Delawen]

My original idea was to extend it later to have a more detailed summary on what type of memory we are using.

But maybe we don't need that kind of detail here?

[Delawen]

We can implement the script to run with different options on what to measure and depending on those parameters, it will impact performance more or less.

But if we are comparing under the same conditions, performance impact is not that important?

[Delawen]

New commit.

Added basic CPU average usage.

Now we have the -m flag to capture more complex memory details. Those details are not being summarized here, I'm working on another script to parse the log properly. But the data is captured every second in case we want to search the logs already.

If we want to use podman to get basic memory statistics and don't burden the java app with more load, we just use the script without that flag.

The -h flag also exists to show basic help.

$ ./measure-boot.sh -m
38a6263f344dc920aa4de8b4093bf93bd321b2528918d9a3ec54c12310968e42
Container 'jdk23-leyden-build' completed bootstrap in 9591 milliseconds
It used the following CPU:
CPU average: 140.75%
It used the following native memory:
Total: reserved=1991MB, committed=162MB
c03b4978b4e1b5e8a712ad70db7c0a8a2cf653d8cf1d1d1dbc8ad173dec339aa
Container 'fedora-standard' completed bootstrap in 10645 milliseconds
It used the following CPU:
CPU average: 177.92%
It used the following native memory:
Total: reserved=1941MB, committed=130MB
f3febe4d1e2ef6556f2861020a2eee615ad29a7a1da33d453429184546cc1fab
Container 'jdk23-build' completed bootstrap in 15353 milliseconds
It used the following CPU:
CPU average: 194.06%
It used the following native memory:
Total: reserved=1943MB, committed=135MB

1ca1dc8349adb1137300e14a097b08936b0fc6b43e2e09793fbaa4407b9d9247
Container 'jdk23-leyden-build' completed bootstrap in 10443 milliseconds
It used the following CPU:
CPU average: 152.20%
It used the following native memory:
Memory limit: 2097152000bytes Memory usage: 98.54MiB / 1.953GiBbytes
57c9a18c8c881ca91778b3af38e984038c772794689d55c7e223fa0cf7cc74d0
Container 'fedora-standard' completed bootstrap in 11351 milliseconds
It used the following CPU:
CPU average: 152.42%
It used the following native memory:
Memory limit: 2097152000bytes Memory usage: 60.1MiB / 1.953GiBbytes
Memory limit: 2097152000bytes Memory usage: 78.98MiB / 1.953GiBbytes
43a401b7ab528f4f4569cf9e7bb1194bb106b6bf32876eca915ed678bad95270
Container 'jdk23-build' completed bootstrap in 14422 milliseconds
It used the following CPU:
CPU average: 197.51%
It used the following native memory:
Memory limit: 2097152000bytes Memory usage: 82.34MiB / 1.953GiBbytes

[franz1981]

why not using the jcmd I sent which just focus on RSS?

[Delawen]

Because the idea is to gather more statistics to offer a better summary later.

[franz1981]

IDK if is too advanced, but we could attach perf stat for the relevant period of time, getting few metrics, but probably we need to add --security-opt seccomp=unconfined if we need to run it from within the container, while if we do it from outside we need to intercept the real PID of the quarkus application first.

This will add few counters, including the AVGCPU but not only...

[Delawen]

IDK if is too advanced, but we could attach perf stat for the relevant period of time, getting few metrics, but probably we need to add --security-opt seccomp=unconfined if we need to run it from within the container, while if we do it from outside we need to intercept the real PID of the quarkus application first.

This will add few counters, including the AVGCPU but not only...

I would leave that for another PR. But sounds like a good idea.