Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Sarath Lakshman
    @t3rm1n4l
    Tried increasing arenas to 8. Still seeing the same periodic lock contention pattern
    Qi Wang
    @interwq
    With >400 threads, you can go more aggressive. Try 64 or even higher.
    Sarath Lakshman
    @t3rm1n4l
    With uniform allocation pattern, I am thinking that all threads are hitting same cycle point and all of them are doing some activity at same time.
    Did not see any improvement moving from 1->8
    Sarath Lakshman
    @t3rm1n4l
    Tried 64 arenas, still the same
    David Goldblatt
    @davidtgoldblatt
    What do the stats look like there?
    Michael Eisel
    @michaeleisel
    Hi, I've built jemalloc on =Mojave with "./configure --enable-debug --disable-zone-allocator" and i get an assertion error at assert(!dependent || leaf != NULL); in rtree_child_leaf_tryread. I've heard this may be due to heap corruption? But asan and guard malloc don't pick up anything. Any ideas what to do next?
    David Goldblatt
    @davidtgoldblatt
    That assert is basically saying "this is a pointer that jemalloc gave out"
    I.e. it's getting a free for a pointer it doesn't know about
    Michael Eisel
    @michaeleisel
    Do you think this is due to heap corruption? I don't explicitly do anything with jemalloc, I just let it override C++ functions like new
    David Goldblatt
    @davidtgoldblatt
    My guess is that a pointer from a zone-managed allocator is leaking into jemalloc
    It could be an application bug or a bug in our zone machinery
    What's the stack trace at the assert?
    Michael Eisel
    @michaeleisel
    Hmm, it looks like even when I don't use --disable-zone-allocator, the constructor function zone_register is not getting called
    Michael Eisel
    @michaeleisel
    Here's the stack (although I get a different crashing stack if I disable asan): https://gist.github.com/michaeleisel/b3d19ae076e293c10699b1f731bbcc54
    David Goldblatt
    @davidtgoldblatt
    Are you able to figure out the allocation site of the crashing allocation?
    I.e. is it yours, a libraries, etc.?
    Michael Eisel
    @michaeleisel
    it's in a destructor for some code of mine
    David Goldblatt
    @davidtgoldblatt
    And that object is definitely operator new'd?
    (I.e. no chance of malloc/delete mismatches, etc.)
    Michael Eisel
    @michaeleisel
    Ah hmm, it appears to have be argv[0]!
    But I still wonder if a C constructor function would beat it to allocation
    David Goldblatt
    @davidtgoldblatt
    It's argv[0] that's being freed?
    Michael Eisel
    @michaeleisel
    Got it now, since I was using a static lib, the linker wasn't including je_zone_register, so using -all_load fixed the problem. Although I also had to turn off asan, since asan messes with things too!
    Thanks @davidtgoldblatt
    David Goldblatt
    @davidtgoldblatt
    np!
    Michael Eisel
    @michaeleisel
    Hi, I was thinking about replacing malloc with jemalloc across the entire operating system. Maybe this is a terrible idea, but I'd be curious to see if it made things snappier. How terrible is this? What would be the best way to do it?
    E.g., replacing libsystem_malloc.dylib with a jemalloc version
    David Goldblatt
    @davidtgoldblatt
    I wouldn't recommend it
    OS X's libc doesn't really support malloc replacement
    Our interposition mechanism is super super hacky
    And occasionally some internal data structure will change layout or its preconditions or whatnot
    Michael Eisel
    @michaeleisel
    You mean in zone.c? Yes, it is risky... I've also replaced malloc with dyld interposing, and it's scary
    David Goldblatt
    @davidtgoldblatt
    And it breaks
    Michael Eisel
    @michaeleisel
    Actually though, replacing libsystemmalloc could be a nice way of doing it, no?
    Because it wouldn't even need zone.c
    David Goldblatt
    @davidtgoldblatt
    I'm not sure -- I don't know a ton about dynamic linking on macs
    Michael Eisel
    @michaeleisel
    That's fair - I might raise it as a Github issue. Thanks for the appropriate warnings though :)
    David Goldblatt
    @davidtgoldblatt
    np :)
    Michael Eisel
    @michaeleisel
    Hi, how does jemalloc do thread-local setup? When I try to play around with adding a thread-local variable to it, I get stuck in an infinite loop when it calls malloc to allocate memory for variable
    Michael Eisel
    @michaeleisel
    (On OS X btw)
    David Goldblatt
    @davidtgoldblatt
    All the thread-locals live in the tsd_t
    (tsd.h)
    IIRC the first access of a thread-local in a dynamic library allocates with malloc on OS X, which triggers the reentrancy
    The tsd module handles the reentrancy scenarios there
    ghaidinyak
    @ghaidinyak_twitter
    I'm using jemalloc on an embedded platform. What's the workflow to get a symbolize a profile from jeprof and then output the gif/pdf/etc on another box? The current jeprof doesn't seem to want to symbolize a profile.
    Jim Walker
    @jimwwalker
    Has anyone had any success with je_malloc_conf on Windows? We use this to configure and works for our Linux/Mac builds but I noted recently it wasn't being picked up on our Windows build.
    David Goldblatt
    @davidtgoldblatt
    We don’t have the magic-symbol malloc_conf on Windows (the linker model doesn’t allow that kind of global symbol merging)
    Just the environment variable or setting it at configure time
    Jim Walker
    @jimwwalker
    thanks, will update my ticket and change course :D