Can anybody help?
This is with MALLOC_CONF="narenas:2,abort_conf:true,dirty_decay_ms:1000,muzzy_decay_ms:1000" INFO ton_indexer_test = RSS: 325.3 MB, Allocated: 305.6 MB, Metadata: 11.4 MB INFO ton_indexer_test = Mapped: 334.0 MB, Active: 315.2 MB, Retained: 418.3 MB This is with no configs at all. INFO ton_indexer_test_mi = RSS: 1.1 GB, Allocated: 456.3 MB, Metadata: 27.5 MB INFO ton_indexer_test_mi = Mapped: 1.2 GB, Active: 496.9 MB, Retained: 2.3 GB
It's rust application using rocks db. Both are using jemalloc.
sizes from heaptrack
Btw, mimalloc is using 2Gb and keep growing.
opt.stats_printto true, you can see the state of the individual arenas at exit, that might help you understand the differences
What is the basic reason of fragmentation? Small values? Maybe there is a guide/doc for tuning it?
I've just set
INFO ton_indexer_test_mi = RSS: 1022.3 MB, Allocated: 501.1 MB, Metadata: 24.7 MB INFO ton_indexer_test_mi = Mapped: 1.1 GB, Active: 526.2 MB, Retained: 974.3 MB
So it's something with decay times
What is the basic reason of fragmentation? Small values?
You want to allocate someting less than a page - e.g. 16B. jemalloc allocates memory using pre-defined sizes classes - i.e. all 16B allocations will come from a set of (4kB) pages; 32B allocations will come from a different set. That makes the maths to allocate / free allocations cheap and quick.
configure: WARNING: unrecognized options: --enable-perf.
We recommend the newest version. The only thing to make sure is, when using jemalloc 5.x, the JVM needs to include this fix: https://bugs.openjdk.java.net/browse/JDK-8215355
(It fixes a bug triggered often with jemalloc 5.x)
It looks the patch has been backported to JDK 11.0.7
Hi all. I'm trying to track down some native memory issues in a java app running on macOS (Big Sur). I was hoping that jemalloc could help, but while i can build it it doesn't seem to give me any heap profile info when i'm running.
I'm following https://github.com/jeffgriffith/native-jvm-leaks/blob/master/README.md as a guide, replacing the .so with the .dylib mac produces.
Searching around, i see there's an old but still open issue suggesting that heap profiling just isn't implemented on MacOS: jemalloc/jemalloc#26 . Am I just out of luck here, and not going to be able to use jemalloc to dig into this? Or have i just failed to link it in properly at runtime or something? (Not sure how to check if the running java process picked up the linked library)
:/ Well, i've got a bit further on loading the libraries. I was doing it wrong, but now have the right DYLD_INSERT_LIBRARIES env var set. Sadly, it's now failing to load them, because of signing issues. Apparently the jdk i'm running is a 'hardened process', and it complains
dyld: warning: could not load inserted library '/usr/local/lib/libjemalloc.dylib' into hardened process because no suitable image found. Did find: /usr/local/lib/libjemalloc.dylib: code signature in (/usr/local/lib/libjemalloc.dylib) not valid for use in process using Library Validation: mapped file has no Team ID and is not a platform binary (signed with custom identity or adhoc?)
after signing them with
Anyone got a clue where to get a non-hardened java runtime that i can use? Sadly all of this is not running in Xcode, because reasons, but i'll see if i can get macOS instruments going?