saudet on sa_gym
RL4J: Use Py_AddPath() instead … (compare)
@sskmaestro Yes, what I meant you have to cache it by yourself on that folder, i.e., download it and put it there. For example this link should have an iris data folder: https://archive.ics.uci.edu/ml/datasets/Iris
I'm so sad Java Developers and tools for Machine Learning always seem just like they're being abandoned.
Hi, I'm debugging an application where physical memory usage grows indefinitely, resulting in OutOfMemory errors. The Errors may occur in a few places, but the ultimate result is "Physical memory usage is too high: physicalBytes(40238M) > maxPhysicalBytes(40050m). It usually takes around 1 million requests to a tomcat app where the ComputationGraphs and DataSetIterator live.
I cannot debug the above application on a linux machine where I can easily change code and iterate quickly. I have created a toy app based on one of the dl4j examples. There is no tomcat, it just loads the net built in CnnSentenceClassificationExample (I used the same training code there), then loops over the DataISetIterator and call net.output on each batch.
I noticed that if logging Pointer.physicalBytes(), in the toy app it goes up until a certain point. Then there is a few second pause, and the totalBytes number drops. Once I see this happen, I see that it happens each time the value gets to a certain max value.
In the tomcat app logs, I don't see the value ever go down. I wasn't logging physical bytes when I saw the OOM, and will do more tests. But, I'm currently trying to pin down where in the dl4j, nd4j, javacpp code is running during this pause (which appears to be garbage collection for native memory). I'd like to find that to see if for some reason that same code segment is being skipped in my tomcat app. Any advice here would be appreciated, thanks.