These are chat archives for opal/opal

12th
Sep 2018
Dan Allen
@mojavelinux
Sep 12 2018 04:29
@Mogztter when I run Antora on large sites, it's necessary to increase the old heap size (to 2048 or 4096). While looking into the problem with the node heap inspector, it seems like once Opal creates an instance of a class, it never lets go of it. I can access instances of classes at the very end of the pipeline even though they are only referenced by variables that are scoped to a function. That's because they are accessible via the Opal object. (for example, via Opal.Antora.const.ConverterExtension.const.ConverterExtension.included_in). This doesn't seem very ideal because it means the application will only continue to consume memory as it grows. Are you aware of this and have you given it any thought?
or am I way off?
perhaps it's how I'm defining classes?
Dan Allen
@mojavelinux
Sep 12 2018 04:43
if I run the following, I can keep the memory from ever increasing (though then certain things break)
delete Opal.Asciidoctor
delete Opal.$$.Asciidoctor
I'd love to know how to remove the instances without blowing away the class definitions
Dan Allen
@mojavelinux
Sep 12 2018 04:49
basically, I need a way to get back to the initial state after require
otherwise, memory just gets depleted over time
Dan Allen
@mojavelinux
Sep 12 2018 05:19
hmm, it does seem to be related to how I'm extending the objects...so perhaps not a general Opal issue
Elia Schito
@elia
Sep 12 2018 07:40
@mojavelinux which opal version (my guess is 0.11 or less)? I think this could happen when extend is used on instances and they get added to the included_in list which is attached to the module itself, hence the leak
Dan Allen
@mojavelinux
Sep 12 2018 08:15
I think I got to the bottom of it. it could still be a bug in opal
but at least I understand it now
the problem was that I was taking a runtime object and calling $extend() on it
when I did that, it was tying local variables into the global Opal object
as soon as I get a chance, I'll set up a little test case
but it's pretty easy to see when using trace-gc
the old heap just kept going up and up and up and never down
so exactly what you said, @elia
:+1:
what makes me very excited about this discover is that it proves that Opal is otherwise performing very well under extreme load
Elia Schito
@elia
Sep 12 2018 08:18
yay!
Dan Allen
@mojavelinux
Sep 12 2018 08:18
I'm using Antora to build an 5.5 GB website that's loaded into memory. Opal is used by Asciidoctor to convert the HTML parts. and with this fix I just made, the memory is stable at about 200MB, which is pretty acceptable
i'm actually kind of amazed that node manages the memory as well as it does
Elia Schito
@elia
Sep 12 2018 08:19
yeah, that leak is known, and was a trade off that doesn't hurt much when dealing with classes/modules (that are global constants), I looked into Weak* JS stuff but there's nothing that works
Dan Allen
@mojavelinux
Sep 12 2018 08:19
even better news is that there was no reason to be doing it the way I was doing it. just sort of naive code
instead of just making a class, I was weaving in a module at runtime. so it's actually better just to extend the class anyway
so an opportunity for an improvement in my book
thanks for the insight!
Elia Schito
@elia
Sep 12 2018 08:23
:D
I need to start putting some testimonials in the opal home page
ylluminate
@ylluminate
Sep 12 2018 14:18
yes, that's a really good idea @elia. people want to know where and how it's being used - otherwise they still believe it's just a toy
Simon George
@sfcgeorge
Sep 12 2018 15:03
Mmm, you can certainly mention any company using Hyperloop like CatPrint, JobLab, and others.