Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 30 2019 21:47
    performantdata commented #113
  • Nov 30 2019 04:34
    performantdata commented #113
  • Nov 30 2019 03:47
    performantdata commented #113
  • Apr 08 2019 17:53

    densh on master

    Scala 2.12 support and updated … Merge pull request #115 from lo… (compare)

  • Apr 08 2019 17:53
    densh closed #115
  • Apr 08 2019 17:52
    densh commented #115
  • Mar 28 2019 20:03
    lolgab commented #113
  • Mar 28 2019 16:02
    lolgab opened #115
  • Feb 11 2019 14:24
    xeno-by removed as member
  • Apr 06 2018 15:30
    satabin commented #113
  • Apr 06 2018 15:22
    satabin commented #113
  • Sep 15 2017 09:01
    daron666 commented #113
  • Jan 24 2017 22:31
    l15k4 commented #47
  • Jan 24 2017 22:18
    l15k4 commented #114
  • Jan 24 2017 22:18
    l15k4 commented #114
  • Jan 24 2017 22:10
    l15k4 closed #114
  • Jan 24 2017 22:10
    l15k4 commented #114
  • Jan 24 2017 22:07
    l15k4 commented #114
  • Jan 24 2017 21:52
    l15k4 commented #114
  • Jan 24 2017 19:49
    DarkDimius commented #114
jeroentervoorde
@jeroentervoorde
Thanks, i will try to find some time this week.
jeroentervoorde
@jeroentervoorde
Hi @densh . i think i managed to create a reproducer: https://github.com/jeroentervoorde/offheap-error-reproducer . Odd thing is that when moving the code to a different file or package it compiles, when moving the code back to the original file the bug also disappeared. I hope the bug is triggered on your machine. I did try it using 2 laptops.
Denys Shabalin
@densh
@jeroentervoorde Thanks for the reproduction!
Jakub Liska
@l15k4
Hey, pls help me understand, why does this snippet needs at least -Xmx3400mto not throw OOM exception ? https://gist.github.com/l15k4/89d79c0acbf98f86904c190d35283f1f
in theory, there is just one object scala.array on heap and it should be Int.MaxValue big, so smth. like -Xmx2300m should suffice, right ?
Denys Shabalin
@densh
Offheap allocations are not GC managed but they still count toward hotspot memory limit as we use sun.misc.Unsafe.allocateMemory under the hood.
It’s similar to the behaviour of direct byte buffers in nio.
Jakub Liska
@l15k4
I'm talking abotu on-heap here
it just requires more heap memory than it should
as besides from the scala.Array everything should be off-heap
Denys Shabalin
@densh
Every time you do offheap.Array.fromArray("foobar".getBytes) you allocate without freeing.
You should either do a manual free or use a region allocator inside the loop.
object Play extends App {

  import scala.offheap
  implicit val alloc = offheap.malloc

  val arr = new scala.Array[offheap.Array[Int]](Int.MaxValue/4)

  var counter = 0
  while (counter < Int.MaxValue/4) Region { implicit alloc =>
    arr(counter) == offheap.Array.fromArray("foobar".getBytes)
    counter+=1
  }

}
This should be better.
Jakub Liska
@l15k4
hmm, interesting riddle, just to be clear, it is not about pace of Garbage Collection of "foobar".getBytes arrays, right ?
it's about the Region Pool using different way of allocation
Denys Shabalin
@densh
Offheap allocations are not garbage-collected. If you use malloc allocator it requires you to free manually.
Region allocator frees everything automatically as soon as region is over.
Matthew de Detrich
@mdedetrich
@l15k4 If yo think about it, its the equivalent of using malloc/free in C, unless you use a region allocator (which will track this for you)
Jakub Liska
@l15k4
I think I get it, I think that you meant that JVM still tracks sun.misc.Unsafe.allocateMemory allocations and that has some memory overhead
the only think unclear to me now is that after your snippet ends, you cannot use that arr anymore, because its elements have been deallocated right away
so it was just for an explanatory purpose, to show that without the allocation overhead it would need less -Xmx memory, right?
Jakub Liska
@l15k4
hm, I still don't understand, why sun.misc.Unsafe.allocateMemory would yield any other on-heap memory overhead besides 8 bytes for the address, which is stored in the array value classes ... from what I can see, every offheap.Array allocation takes 8+8 for some reason
Jakub Liska
@l15k4
it is all clear now densh/scala-offheap#114
Jakub Liska
@l15k4
I just didn't know that Value Classes get boxed when stored to Array
Kaspar Fischer
@hbf
Is this project still active? Or has development moved to another project?
Denys Shabalin
@densh
@hbf The project has been “on hold” for quite some time now.
I don’t really have time to move it forward, if anyone is interested in stepping in please let me now.
Kaspar Fischer
@hbf
@densh Thanks for the clarifications, much appreciated!