Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    mkeskells
    @mkeskells
    :-)
    Guillaume Martres
    @smarter
    Harrison Houghton
    @hrhino
    The compiler-benchmark job seems to be sad...
    whatever volume influx uses appears to be out of memory
    and the scala corpus (for 2.13.x) may need updated so that the added method ioobe on Statics exists
    Jason Zaugg
    @retronym
    Thanks for the report, @hrhino . @adriaanm, I can’t ssh to jenkins-grafana to remedy…
    Jason Zaugg
    @retronym
    I’ve freed up some disk space, lets see if this works:https://scala-ci.typesafe.com/view/scala-bench/job/compiler-benchmark/2103/console
    Guillaume Martres
    @smarter
    So Martin added an instrumentation phase to the compiler to count allocations and used it to reduce the allocation rate of the compiler itself a bit: lampepfl/dotty#5748
    This seems to help at least some benchmarks: http://dotty-bench.epfl.ch/ (look at the drop around 19/1 in Vector, scalapb, scalap, ...)
    I wish there was some reliable way to count allocations that don't get elided by escape analysis
    Guillaume Martres
    @smarter
    hmm, I wonder if we could call https://docs.oracle.com/javase/8/docs/jre/api/management/extension/com/sun/management/ThreadMXBean.html#getThreadAllocatedBytes(long) before and after each allocation and assume it was escaped if the difference is zero
    Ichoran
    @Ichoran
    I haven't used that particular interface before, but I've not found that the other instrumentation is very good at counting small differences in bytes allocated. It seemed more chunked to me in the past.
    Yeah, the API says "The returned value is an approximation because some Java virtual machine implementations may use object allocation mechanisms that result in a delay between the time an object is allocated and the time its size is recorded."
    Guillaume Martres
    @smarter
    so it might or might not work :)
    it seems to be rather precise from my limited experimentation
    (on openjdk 8)
    Ichoran
    @Ichoran
    Well, if it works in practice, that's cool. I just can report that with similar APIs I've found it to not work as of I guess about 6 years ago.
    mkeskells
    @mkeskells
    The allocation counters is not chunked but it is in too expensive to call for each allocation. It is sampled in the the compiler if you enable profiling and then problem down by phase
    Guillaume Martres
    @smarter
    so I played with it and I got "48" as the minimum value between two calls to that method without allocating anything
    so I instrumented every allocation site in Dotty but even when the compiler is hot, only three call sites show up with "48" as the allocation size
    might be because all the instrumentation makes the JIT give up, or because even when something is elided, the fields of the elided objects might still be allocated
    mkeskells
    @mkeskells
    have a look in trait AllocationTest
    junit helpers and calibration for the delta between call
    the difference between successive calls is VM specific I recall
    Guillaume Martres
    @smarter
    interesting, thanks
    mkeskells
    @mkeskells
    its used in IndexedSeqTest - ther used to be some allocations in Vector#drop etc used to create Iteratorsto compare
    Eric Peters
    @er1c
    Is there a way to specify JVM settings other than the default VM options: -Xms2G -Xmx2G -Xss2M
    Eric Peters
    @er1c
    think I got it: cold -psource=@/Users/eric/dd/bigdata-SA-37745-timezone/web_api/target/web-api-test.args -jvmArgs "-Xms1024m -Xmx4096m -Xss256m -XX:+UseG1GC -XX:MaxMetaspaceSize=512m -XX:ReservedCodeCacheSize=256m -Djava.security.egd=file:///dev/urandom -Dfile.encoding=UTF-8 -Doracle.jdbc.mapDateToTimestamp=false"
    mkeskells
    @mkeskells
    @er1c for what sort of test?
    Eric Peters
    @er1c
    Trying to change the garbage collector to see what impact that has, I got a tip to use compiler-benchmark to try and test our repo compilation speed by changing the VM args around (more memory, GC, etc)
    not sure if that is realistic I guess, but fun to play with, the other option is to just run a "sbt clean;test:compile" 10 times in a row in a Jenkins job and to try and manually compare changes
    the new sbt 1.4.x has a warning on our code that says we are hitting GC limits which is a nice warning too
    is this the wrong tool to use?
    mkeskells
    @mkeskells
    If you can use Graal it will probably be a lot faster. I havent tried other options
    good luck!