by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Miles Sabin
    @milessabin
    Is there any policy for when benchmarks get merged into the mainline?
    Jason Zaugg
    @retronym
    No policy, I’m very happy to have a variety of workloads
    The question would be whether we start tracking performance of a new benchmark regularly or not
    Jason Zaugg
    @retronym
    "The curious case of the abrupt compiler slowdown" https://github.com/scala/scala-dev/issues/518#issuecomment-391890188
    Performance engineering never gets boring.
    Ichoran
    @Ichoran
    That is a weird one.
    The only place I've seen that kind of thing before is with deoptimization of a hot loop handling a rare case.
    Jason Zaugg
    @retronym
    Here's how to use a microbenchmark to exercise part of scalac's internals, and use JMH profilers to look at changes in runtime and allocation rate: https://github.com/scala/scala/pull/6592#issuecomment-392665745
    We decided that those sort of benchmarks belong in scala/scala, the compiler-benchmark project will just focus on the "macro" benchmarks of compiling our test corpi and pushing results into Grafana.
    Rory Graves
    @rorygraves
    Are there a good set of benchmarks for playing with Champmap anywhere!
    ? Rather than !
    Lukas Rytz
    @lrytz
    there is some discussion and some code in scala/collection-strawman#342, but I don't know details. maybe @julienrf?
    Rory Graves
    @rorygraves
    Are there any equvilents for Map?
    Julien Richard-Foy
    @julienrf
    No. You can easily copy this one and adapt it for Maps, I guess.
    Rory Graves
    @rorygraves
    Will do. Thanks
    Miles Sabin
    @milessabin
    @retronym this CurryOn talk was pretty terrifying: https://youtu.be/hj4VmvyqbKY
    Jason Zaugg
    @retronym
    @milessabin watching, thanks for the link. We looked at that paper and associated best-practices for machine config when setting up our benchmark server. https://github.com/scala/scala-dev/issues/338#issuecomment-287980632
    Jason Zaugg
    @retronym
    On using JITWatch to analyse escape analysis: https://github.com/scala/scala-dev/issues/498#issuecomment-407241945
    Miles Sabin
    @milessabin
    @retronym understood, but the conclusions, including from the after talk chat, seemed pretty dismal.
    Jason Zaugg
    @retronym
    Non determinism of VM isn’t great for benchmarking, but stepping back from that particular problem, I believe we get better peak performance overall as a result, especially in real world workloads.
    The paper calls out counterexamples where JIT optimisations are counterproductive, but I don’t think it claims these are representative
    I do think it is necessary to treat the VM as whitebox if you care about performance: understand its internals, interpret benchmark results together with profiles, JIT log analysis, GC metrics, etc
    Jason Zaugg
    @retronym
    Epic 2.12.x performance backport PR: scala/scala#7147
    Ghost
    @ghost~54a283bfdb8155e6700e3f59
    😍
    mkeskells
    @mkeskells
    :-)
    Guillaume Martres
    @smarter
    Harrison Houghton
    @hrhino
    The compiler-benchmark job seems to be sad...
    whatever volume influx uses appears to be out of memory
    and the scala corpus (for 2.13.x) may need updated so that the added method ioobe on Statics exists
    Jason Zaugg
    @retronym
    Thanks for the report, @hrhino . @adriaanm, I can’t ssh to jenkins-grafana to remedy…
    Jason Zaugg
    @retronym
    I’ve freed up some disk space, lets see if this works:https://scala-ci.typesafe.com/view/scala-bench/job/compiler-benchmark/2103/console
    Guillaume Martres
    @smarter
    So Martin added an instrumentation phase to the compiler to count allocations and used it to reduce the allocation rate of the compiler itself a bit: lampepfl/dotty#5748
    This seems to help at least some benchmarks: http://dotty-bench.epfl.ch/ (look at the drop around 19/1 in Vector, scalapb, scalap, ...)
    I wish there was some reliable way to count allocations that don't get elided by escape analysis
    Guillaume Martres
    @smarter
    hmm, I wonder if we could call https://docs.oracle.com/javase/8/docs/jre/api/management/extension/com/sun/management/ThreadMXBean.html#getThreadAllocatedBytes(long) before and after each allocation and assume it was escaped if the difference is zero
    Ichoran
    @Ichoran
    I haven't used that particular interface before, but I've not found that the other instrumentation is very good at counting small differences in bytes allocated. It seemed more chunked to me in the past.
    Yeah, the API says "The returned value is an approximation because some Java virtual machine implementations may use object allocation mechanisms that result in a delay between the time an object is allocated and the time its size is recorded."
    Guillaume Martres
    @smarter
    so it might or might not work :)
    it seems to be rather precise from my limited experimentation
    (on openjdk 8)
    Ichoran
    @Ichoran
    Well, if it works in practice, that's cool. I just can report that with similar APIs I've found it to not work as of I guess about 6 years ago.
    mkeskells
    @mkeskells
    The allocation counters is not chunked but it is in too expensive to call for each allocation. It is sampled in the the compiler if you enable profiling and then problem down by phase
    Guillaume Martres
    @smarter
    so I played with it and I got "48" as the minimum value between two calls to that method without allocating anything
    so I instrumented every allocation site in Dotty but even when the compiler is hot, only three call sites show up with "48" as the allocation size
    might be because all the instrumentation makes the JIT give up, or because even when something is elided, the fields of the elided objects might still be allocated
    mkeskells
    @mkeskells
    have a look in trait AllocationTest
    junit helpers and calibration for the delta between call
    the difference between successive calls is VM specific I recall
    Guillaume Martres
    @smarter
    interesting, thanks
    mkeskells
    @mkeskells
    its used in IndexedSeqTest - ther used to be some allocations in Vector#drop etc used to create Iteratorsto compare