These are chat archives for deeplearning4j/deeplearning4j/earlyadopters

8th
May 2016
Samuel Audet
@saudet
May 08 2016 06:29
@agibsonccc late to the party, sorry. right, Gradle and the rest. I'm thinking we can create one more artifact like nd4j-native-lib, that would depend on both nd4j-native and the classifier. Then it wouldn't break Gradle, sbt, etc but to get things automated there we would need plugins: bytedeco/javacv#395
Any
Anyway, having an optional auto include artifact like that would also make it easy for users to switch manually to the -windows-x86_64-mkl or whatever other "platform" we would build for
Patrick Skjennum
@Habitats
May 08 2016 08:33
cpu utilization is down by a lot since i pulled yestarday
like 50% now
raver119
@raver119
May 08 2016 08:33
hm...
Patrick Skjennum
@Habitats
May 08 2016 08:33
it was flat 100% for the same task last week
raver119
@raver119
May 08 2016 08:33
might be my fault
can i see hot spots?
Patrick Skjennum
@Habitats
May 08 2016 08:38
blob
blob
just to show i have more than enough memory
blob
raver119
@raver119
May 08 2016 08:40
hm, my changes might affect native changes in some situations, but definitely can’t cause more allocations on cpu version
however, i’ll push some changes for my changes, just to check if that’s me or not
Patrick Skjennum
@Habitats
May 08 2016 08:40
it's allocating like crazy
Adam Gibson
@agibsonccc
May 08 2016 08:41
Hmm pairwise has been a frequent problem
Memset is known
Patrick Skjennum
@Habitats
May 08 2016 08:42
you made me forecibly update javacpp to 1.2 yestarday, could that be an issue?
i updated to master on javacpp
Adam Gibson
@agibsonccc
May 08 2016 08:43
No
Post an issue with the perf label though
Libnd4j repo
Patrick Skjennum
@Habitats
May 08 2016 08:49
done
want me to include more info?
@agibsonccc i can't put tags
Adam Gibson
@agibsonccc
May 08 2016 08:55
@Habitats thanks a lot
Alex got it
Patrick Skjennum
@Habitats
May 08 2016 09:13
(y)
Jeroen Steggink
@jsteggink
May 08 2016 09:50
Latest version gives the following exception:
Exception in thread "main" java.lang.UnsatisfiedLinkError: no jnind4j in java.library.path
Caused by: java.lang.UnsatisfiedLinkError: no nd4j in java.library.path
Anything I have to do?
Adam Gibson
@agibsonccc
May 08 2016 09:51
@jsteggink full github gist please?
How are you trying to build it?
Adam Gibson
@agibsonccc
May 08 2016 09:53
Are you running it via intellij?
or command line?
Jeroen Steggink
@jsteggink
May 08 2016 09:53
Eclipse
Adam Gibson
@agibsonccc
May 08 2016 09:53
hm
yeah...
hmm
It's a runtime configuration of some kind
Jeroen Steggink
@jsteggink
May 08 2016 09:53
I have compiled and runned it almost everyday the last few weeks with almost no problems
Adam Gibson
@agibsonccc
May 08 2016 09:53
I'm not sure why the runtime configurationwouldn't have picked it up
Try reinstalling ia mvn clean install on lib and nd4j?
Jeroen Steggink
@jsteggink
May 08 2016 09:54
I just did
Adam Gibson
@agibsonccc
May 08 2016 09:54
@saudet any thoughts on this with eclipse?
Jeroen Steggink
@jsteggink
May 08 2016 09:54
I can try and run it from commandline
Adam Gibson
@agibsonccc
May 08 2016 09:54
I'm really not sure. None of us use eclipse
yeah that'd be great
maybe bundle it as an uber jar (maven shade plugin ring a bell?)
hmm
Command line works fine though
Adam Gibson
@agibsonccc
May 08 2016 10:07
@jsteggink Could you file an issue?
This is great information
thanks
libnd4j/issues
Jeroen Steggink
@jsteggink
May 08 2016 10:07
I will, no problem
It wouldn't take long
Jeroen Steggink
@jsteggink
May 08 2016 10:07
not nd4j issues?
Adam Gibson
@agibsonccc
May 08 2016 10:11
libnd4j for c++ related
Jeroen Steggink
@jsteggink
May 08 2016 10:11
Ah never mind, I fixed it myself. Stupid me didn't update the dependencies in Eclipse. Sorry about that
Adam Gibson
@agibsonccc
May 08 2016 10:11
@jsteggink still file an issue?
At least mention that
that's still a gotcha for the docs
thanks!
still really helpful
that's at least a troubleshooting thing
Jeroen Steggink
@jsteggink
May 08 2016 10:11
Yeah, I guess this only happens when you build it yourself
Adam Gibson
@agibsonccc
May 08 2016 10:11
none of use eclipse
Jeroen Steggink
@jsteggink
May 08 2016 10:12
Maybe I should write a help on how to use Eclipse
Adam Gibson
@agibsonccc
May 08 2016 10:12
we're going to allow people to dynamically link against new/old versions of libnd4j
Jeroen Steggink
@jsteggink
May 08 2016 10:12
Alright, then it will happen definitely
Adam Gibson
@agibsonccc
May 08 2016 10:12
@jsteggink that would be godly
seriously
even just some basics
eclipse is harder to deal wth :(
I know some use it though
Jeroen Steggink
@jsteggink
May 08 2016 10:13
Where would I put 3.9 stuff?
Adam Gibson
@agibsonccc
May 08 2016 10:13
we just can't test every possible configuration
Jeroen Steggink
@jsteggink
May 08 2016 10:13
That's true
Adam Gibson
@agibsonccc
May 08 2016 10:14
this is deeplearing4j.org
Jeroen Steggink
@jsteggink
May 08 2016 10:14
That's what the community is for :)
this is nd4j.org
for libnd4j compilation info specifically
just put it in an eclipse.md
in the main libnd4j repo
and send us a PR
Jeroen Steggink
@jsteggink
May 08 2016 10:15
Ok, I'll try and add some Eclipse stuff
Adam Gibson
@agibsonccc
May 08 2016 10:15
We'll get a page compiled with libnd4j on nd4j.og later on
Jeroen Steggink
@jsteggink
May 08 2016 10:15
alright
Adam Gibson
@agibsonccc
May 08 2016 10:15
We're just makign it easy right now
thanks man
seriously helpful
Jeroen Steggink
@jsteggink
May 08 2016 10:16
no problem
Adam Gibson
@agibsonccc
May 08 2016 10:16
:D
Ruben Fiszel
@rubenfiszel
May 08 2016 10:54
Hello, haven't updated for a week. I changed my code a lot so it might be my fault but it seems unlikely
I have this error on Nd4j.create(ar[][]) where ar is 50x52
The error is here apparently
Ruben Fiszel
@rubenfiszel
May 08 2016 11:01
ok nevermind it was my fault
:D
Paul Dubs
@treo
May 08 2016 11:01
how did you get this part of code to misbehave?
Ruben Fiszel
@rubenfiszel
May 08 2016 11:02
The inners arrays are not of the same length
I know, impressively wrong :-°
Ruben Fiszel
@rubenfiszel
May 08 2016 11:07
Reason is that I create the array this way ll.map(_.toArray) where ll is a list of list and the inner lists are not of the same size
Paul Dubs
@treo
May 08 2016 11:08
I know it isn't pure and functional, but why go through all the hassle, if you could simply double for loop through it and put it into the ndarray directly?
Ruben Fiszel
@rubenfiszel
May 08 2016 11:13
@treo Because I don't know the performance of the underlying structure so it might not be sure that this way is more efficient for me. But I can do it if you know that there is a benefit
Paul Dubs
@treo
May 08 2016 11:14
you have just seen what the constructor does
Ruben Fiszel
@rubenfiszel
May 08 2016 11:14
Well now I have indeed :D
@treo one benefit also is that it's much easier to debug. If I had gone for a double loop I would have never caught this error, the loop would have stopped before
Paul Dubs
@treo
May 08 2016 11:16
the error wouldn't have happened at all
Ruben Fiszel
@rubenfiszel
May 08 2016 11:16
Which is terrible
Since the lists being not of the same size means that I did something wrong
Paul Dubs
@treo
May 08 2016 11:16
that is a different problem :P
Ruben Fiszel
@rubenfiszel
May 08 2016 11:17
Yes but it's important to know that I have this problem, else I can't fix it :D
Arthur Rehm
@rehm
May 08 2016 13:06
Hi, i use 0.4-rc3.9-SNAPSHOT Version - since yesterday i get an java.lang.RuntimeException: java.lang.NullPointerException by loading my model with WordVectorSerializer.loadTxtVectors(VectorFile);
```Caused by: java.lang.RuntimeException: java.lang.NullPointerException
at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:4788)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:4716)
at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:148)
... 7 more
Caused by: java.lang.NullPointerException
at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:4749)
... 9 more

Function:

vec = WordVectorSerializer.loadTxtVectors(VectorFile);

Exception:

Caused by: java.lang.RuntimeException: java.lang.NullPointerException
    at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:4788)
    at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:4716)
    at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:148)
    ... 7 more
Caused by: java.lang.NullPointerException
    at org.nd4j.linalg.factory.Nd4j.initWithBackend(Nd4j.java:4749)
    ... 9 more
raver119
@raver119
May 08 2016 13:25
@rehm please use http://gist.github.com for logs
also, please give me full stack trace
raver119
@raver119
May 08 2016 13:30
do you actually have backend in pom.xml ?
show your pom.xml
as gist please
raver119
@raver119
May 08 2016 13:34
yes
remove nd4j-x86 dep
and replace it with nd4j-native
for rc3.9 nd4-x86 isn’t used
Arthur Rehm
@rehm
May 08 2016 13:57
but still get the same java.lang.NullPointerException
Adam Gibson
@agibsonccc
May 08 2016 14:05
@rehm remove this:
<repositories>
<repository>
<id>snapshots-repo</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
<releases><enabled>false</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
make sure to rebuild nd4j as well
Shape info provider is new
Did you forget to instanitate it for cpu?
look at the line the null pointer is occurring on
maybe you forgot to update the nd4j-native.properties with something you were using for gpus only?
it looks like it's a backend not being setup right
I see the shape info provider in both properties
Adam Gibson
@agibsonccc
May 08 2016 14:10
@rehm I'm not sure here..could you try rebuilding nd4j first?
after you remove that repo?
do a git pull etc as well
Arthur Rehm
@rehm
May 08 2016 14:37
@agibsonccc - I rebuild nd4j from source, here the stack trace: https://gist.github.com/rehm/722138a741cb4be9b1f33c3586d75a1f
raver119
@raver119
May 08 2016 14:41
@agibsonccc no, it’s attached the same way as other things in nd4j, using reflection
however, i wonder how properties file is absent, it’s widely used in backend initialization
Jeroen Steggink
@jsteggink
May 08 2016 18:38
Having some problems with NaN scores appearing without any code change. This started about two weeks ago. The funny thing is, whatever dataset, it always happens at a fixed iteration. I tried removing the regularization, but no change. For now, the only thing that works is using Nd4j.ENFORCE_NUMERICAL_STABILITY = true;. Any tips on how to debug this?
Arthur Rehm
@rehm
May 08 2016 20:58
@raver119 - after git pull - 'origin/master' i cant build the nd4j-native from source. https://gist.github.com/rehm/c85463e2029f917450a9f667545454a5
Paul Dubs
@treo
May 08 2016 20:59
take a look further up the log
it tells you what the problem is
raver119
@raver119
May 08 2016 21:00
as @treo said - you've copied wrong portion of error
btw, @treo atm comparison reports 6200 - 6500 ms
without profiler
in profiler +1k ms
Paul Dubs
@treo
May 08 2016 21:00
it is slowly moving in the right direction :)
raver119
@raver119
May 08 2016 21:01
broadcast still the main problem
damn, and the fact that i can't find where exactly i've set restrictions on host memory for shapeBuffers lol
Paul Dubs
@treo
May 08 2016 21:03
lol :D
raver119
@raver119
May 08 2016 21:20
At iteration 5 a single iteration takes 2608 MILLISECONDS
At iteration 10 a single iteration takes 2526 MILLISECONDS
At iteration 15 a single iteration takes 2564 MILLISECONDS
pew
pew-pew-pew\
Paul Dubs
@treo
May 08 2016 21:21
oooooh nice :D
raver119
@raver119
May 08 2016 21:21
yup :)
Paul Dubs
@treo
May 08 2016 21:22
with this cuda is only 2x away from cpu
(if I recall correctly what the cpu timing was)
raver119
@raver119
May 08 2016 21:22
1.6s you've said
so it's less then x2
Paul Dubs
@treo
May 08 2016 21:22
even better :D
raver119
@raver119
May 08 2016 21:23
not a win yet, but significant step
i'll commit my changes now, but won't merge
if you want to play there - pull my branches as usual
Paul Dubs
@treo
May 08 2016 21:23
I'm currently training a net, so I'll probably only get to try it out tomorrow
raver119
@raver119
May 08 2016 21:24
sure, as you wish
hmm...
raver119
@raver119
May 08 2016 21:29
yes, profiling map changed significantly
Paul Dubs
@treo
May 08 2016 21:32
to the tune of 99 bottles of beer on the wall:
99 bottlenecks on the wall, 99 bottlenecks .
Take one down, make it go fast, 98 bottlenecks on the wall...
raver119
@raver119
May 08 2016 21:35
rofl
raver119
@raver119
May 08 2016 21:43
and obviously, async model contains two bugs.
damn
Paul Dubs
@treo
May 08 2016 21:44
to the tune of 99 bottles of beer:
99 bugs on the wall, 99 tiny bugs.
Take one down, fix it up, 101 bugs on the wall...
:P
raver119
@raver119
May 08 2016 21:44
yea, smth like that :/
i definitely don't like the thing i see in profiler.
it's wrong.
i've seriously fckd somewhere there
Alex Black
@AlexDBlack
May 08 2016 23:54
@jsteggink open an issue with something I can run to reproduce that and I'll take a look
Adam Gibson
@agibsonccc
May 08 2016 23:58
@rehm you need to set the LIBND4j_HOME environment variable. it says it right in the error message