Open Event Website Generator https://github.com/fossasia/open-event-wsgen/
just a sec
here it is
on cmd just write "npm start"
redis server was required to be started and it seems it is working as you received pong
we are not supposed to use redis cli
ok
is this supposed to happen?
open-event-wsgen
ok ok
i think its working
@iamareebjamal I am trying to figure out the issue which we were facing when wsgen was hosted on free dyno frompast two days. Here are the things which I have found out :-
- There is a sudden increase in ram usage whenever there is a pingTimeout.
- The wsgen server doesn't respond while the resizing task is running since it is being run on the speaker images directory using the async library (async.each), all theresources get busy and the main thread gets blocked due to which server doesn't respond to any request even the regular interval ping request to maintain the socket connection. Thus the socket disconnects on pingTimeout.
- For the above thing I tried out 2 things, tried increasing the pingTimeout to 60sec from 25sec(default). The issue of timeout get resolved but in the meantime if any other get request comes the main thread doesn't respond. So then I kept the pingTimeout the same and used async.eachLimit with 2 as the limit instead of async.each which had no limit. It solved the issue.
Another thing I have noticed is that, the ram usage is pretty high and it doesn't scale down after one task generation is complete. It remains the same. This thing made me look if there is memory leak in the application. I used --inspect to load the nodejs debugger but the application heap memory do scale up to ~50MB when there is a genaration event but it scales down to normal ~37MB as well when the generation of website is complete.
When a request of website generation is there it hits 300MB and doesn't scale down and keeps scaling up on subsequent requests
I tried searching the web but can't it didn't help.
Please do suggest in what direction should I move on from here
The wsgen server doesn't respond while the resizing task is running since it is being run on the speaker images directory using the async library
That's true for sync, not async
sync blocks while doing I/O, async keeps listening
It's also important that async.each is taking a batched or parallel approach as I wrote in the PR
If it is taking pictures one by one and resizing, then it's not good enough and will take a lot of time
@iamareebjamal I tested this out for my dyno(https://damp-wave-90723.herokuapp.com/). This actually has helped achieve significant improvement in the ram usage as jemalloc is actually effective in resolving this issue for sharp. So jemalloc buildpack(https://github.com/gaffneyc/heroku-buildpack-jemalloc) must be enabled on prod as well as dev heroku dyno as it returns the free memory back to the OS and thus solves the issue of blocked memory.
I'll share it in the morning and I'll raise another PR with some improvements for batch processing of images.
Can you review plz here the link
@iamareebjamal the statistics for the total ram usage(with or without jemalloc as memory allocator) is as follows:
- Peak memory usage while generating website: ~790MiB(without jemalloc) and ~380MiB(with jemalloc)
- Post website generation: ~791MiB(without jemalloc) and ~178MiB(with jemalloc)
Another round of generation caused the dyno to crash(without jemalloc) but the ram usage remained same with jemalloc.
The above results were tested for this event fossasiaSummit2019(https://api.eventyay.com/v1/events/3689bc98)
Thank you @iamareebjamal I have even raised a PR for processing a batch of 15 images at a time parallelly and this way we won't even face denial of service which was faced earlier for about 10-15 secs. Can you please review this fossasia/open-event-wsgen#2411