These are chat archives for deeplearning4j/deeplearning4j/earlyadopters

28th
Jun 2016
Kshitij Jain
@jain98
Jun 28 2016 00:23
Could one of you help me set up my system, without the CUDA backend...Been at it since yesterday and have been getting the same errors
Samuel Audet
@saudet
Jun 28 2016 00:24
What kind of errors?
Kshitij Jain
@jain98
Jun 28 2016 00:25
I'm following the windows setup instructions
firstly, I think I'm building my cpu backend properly
This is what my log for ./buildnativeoperations.sh looks like-

$ ./buildnativeoperations.sh
eval cmake
Running windows
PACKAGING = none
BUILD = release
CHIP = cpu
LIBRARY TYPE = dynamic
/C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu
-- The C compiler identification is GNU 5.3.0
-- The CXX compiler identification is GNU 5.3.0
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/msys64/mingw64/bin/g++.exe
-- Check for working CXX compiler: C:/msys64/mingw64/bin/g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)
ADDING BLAS
-- dir='C:/Users/User2/Desktop/git/github/libnd4j/include'
CPU BLAS
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -fopenmp
-- Could NOT find MKL (missing: MKL_LIBRARY_DIR LIB_MKL_RT LIB_PTHREAD MKL_INCLUDE_DIR)
USING BLAS
-- A library with BLAS API not found. Please specify library location.
-- A library with BLAS API not found. Please specify library location.
-- LAPACK requires BLAS.
ARCH IS x86_64
and PROCESSOR AMD64
32 bit
CMAKE MINGW ROOT /mingw32
-- Could not find OpenBLAS include. Turning OpenBLAS_FOUND off
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:

MKL_MULTI_THREADED

-- Build files have been written to: C:/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu
/C/msys64/mingw64/bin/cmake.exe -H/C/Users/User2/Desktop/git/github/libnd4j -B/C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu --check-build-system CMakeFiles/Makefile.cmake 0
/C/msys64/mingw64/bin/cmake.exe -E cmake_progress_start /C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu/CMakeFiles /C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make1: Entering directory '/C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu'
make -f blas/CMakeFiles/nd4j.dir/build.make blas/CMakeFiles/nd4j.dir/depend
make2: Entering directory '/C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu'
/C/msys64/mingw64/bin/cmake.exe -E cmake_depends "MSYS Makefiles" /C/Users/User2/Desktop/git/github/libnd4j /C/Users/User2/Desktop/git/github/libnd4j/blas /C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu /C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu/blas /C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu/blas/CMakeFiles/nd4j.dir/DependInfo.cmake --color=
Scanning dependencies of target nd4j
make2: Leaving directory '/C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu'
make -f blas/CMakeFiles/nd4j.dir/build.make blas/CMakeFiles/nd4j.dir/build
make2: Entering directory '/C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu'
[ 33%] Building CXX object blas/CMakeFiles/nd4j.dir/cpu/NativeBlas.cpp.obj
cd /C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu/blas && /C/msys64/mingw64/bin/g++.exe -DCPUBLAS=true -Dnd4j_EXPORTS -I/C/Users/User2/Desktop/git/github/libnd4j/include -Wall -O3 -Wl,-rpath,RIGIN/ -fopenmp -Wall -O3 -std=c++11 -fassociative-math -funsafe-math-optimizations -o CMakeFiles/nd4j.dir/cpu/NativeBlas.cpp.obj -c /C/Users/User2/Desktop/git/github/libnd4j/blas/cpu/NativeBlas.cpp
[ 66%] Building CXX object blas/CMakeFiles/nd4j.dir/cpu/NativeOps.cpp.obj
cd /C/Users/User2/Desktop/git/github/libnd4j/blasbuild/cpu/blas && /C/msys64/mingw64/bin/g++.exe -DCPUBLAS=true -Dnd4j_EXPORTS -I/C/Users/User

Samuel Audet
@saudet
Jun 28 2016 00:27
Please put that in the a gist next time
Kshitij Jain
@jain98
Jun 28 2016 00:28
My bad
Could you read it or should I put it in a gist now?
Samuel Audet
@saudet
Jun 28 2016 00:28
I don't see any errors anyway
Kshitij Jain
@jain98
Jun 28 2016 00:29
wait...let me create a gist
You'll see the "Building CXX object blas/CMakeFiles/nd4j.dir/cpu/NativeOps.cpp.obj" is only being completed 66%
Samuel Audet
@saudet
Jun 28 2016 00:31
[100%] Built target nd4j → seems fine to me
Kshitij Jain
@jain98
Jun 28 2016 00:33
ok...so if there is not error here, then I may be getting an error building nd4j
let me create a gist for that
here you go @saudet
Samuel Audet
@saudet
Jun 28 2016 00:37
!!! LIBND4J_HOME must be a valid unix path! → your path needs to look like /c/some/path/to/libnd4j
Kshitij Jain
@jain98
Jun 28 2016 00:38
Yeah, I read that
and I've tried putting it in different places
in the C drive
still giving me this error
Samuel Audet
@saudet
Jun 28 2016 00:40
Where is it now?
Kshitij Jain
@jain98
Jun 28 2016 00:41
C:\Program Files\libnd4j
Samuel Audet
@saudet
Jun 28 2016 00:42
So try to call export LIBND4J_HOME="/c/Program Files/libnd4j/" before the build
Kshitij Jain
@jain98
Jun 28 2016 00:53
ok
Adam Gibson
@agibsonccc
Jun 28 2016 01:05
@saudet yeah I got it
Kshitij Jain
@jain98
Jun 28 2016 01:07
If I'm not building the CUDA backend, will I get this error-
Failed to execute goal on project nd4j-cuda-7.5-platform: Could not resolve dependencies for project org.nd4j:nd4j-cuda-7.5-platform:jar:0.4-rc3.11-SNAPSHOT: Could not find artifact org.nd4j:nd4j-cuda-7.5:jar:0.4-rc3.11-SNAPSHOT in sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots) -> [Help 1]
Samuel Audet
@saudet
Jun 28 2016 01:10
That's fine, you can ignore that or skip it with mvn -pl \!nd4j-cuda-7.5,\!nd4j-cuda-7.5-platform ...
Kshitij Jain
@jain98
Jun 28 2016 01:10
ok, Thanks @saudet
Just another quick question. For using the native backend, we have to make changes to the pom.xml file, right?
Samuel Audet
@saudet
Jun 28 2016 01:12
Sure, just check the one from dl4j-0.4-examples
Justin Long
@crockpotveggies
Jun 28 2016 01:14
@raver119 thank you for the update :)
Kshitij Jain
@jain98
Jun 28 2016 01:16
It wouldn't be possible to not have any nd4j dependencies at all right?
Adam Gibson
@agibsonccc
Jun 28 2016 01:17
you always need n4j
nd4j
That's what the neural nets are written in
Not sure why you'd want to get rid of it
Your other option is straight java where it will take 5 years to train a simple model
Kshitij Jain
@jain98
Jun 28 2016 01:23
so where do I do this?

In order to use your new shiny backends you will have to switch your application to use the version of ND4J that you just compiled and to use the native backend instead of x86.

For this you change the version of all your ND4J dependencies to "0.4-rc3.10-SNAPSHOT".

Adam Gibson
@agibsonccc
Jun 28 2016 01:23
No don't use snapshot
If you're building from source use whatever the version in dl4j master is
Kshitij Jain
@jain98
Jun 28 2016 01:38
Ok, think I may be done setting up. What's the best way to start contributing?
Adam Gibson
@agibsonccc
Jun 28 2016 01:39
Get used to the framework first
Pick something small you want to do
And we'll direct you
From there, get used to libnd4j
I'd inch your way in
Fabiano
@ftarlao
Jun 28 2016 12:09
Hi, I am starting using dl4j. I have a maven java project with Netbeans 8.0.1. Things works well with deedeeplearning4j 0.4-rc3.8 and x86 backend. I have tried using 0.4-rc3.10 with the 'native' backend, comilation works fine but at runtime it fails to scan libraries inside the jar files. The runtime error is as follows:
--- exec-maven-plugin:1.2.1:exec (default-cli) @ MouseAuthenticationANN ---
13:58:57.674 [main] INFO o.d.e.convolution.TraceClassifier - Create dataset....
Number users: 4432
Residual users: 3377
13:58:59.097 [main] DEBUG org.reflections.Reflections - going to scan these urls:
jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-context/0.4-rc3.10/nd4j-context-0.4-rc3.10.jar!/
jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native-api/0.4-rc3.10/nd4j-native-api-0.4-rc3.10.jar!/
jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-api/0.4-rc3.10/nd4j-api-0.4-rc3.10.jar!/
jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-buffer/0.4-rc3.10/nd4j-buffer-0.4-rc3.10.jar!/
jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-common/0.4-rc3.10/nd4j-common-0.4-rc3.10.jar!/
jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/
13:58:59.306 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libgomp.so.1 in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner SubTypesScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libgomp.so.1 in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner TypeAnnotationsScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libgfortran.so.3 in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner SubTypesScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libgfortran.so.3 in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner TypeAnnotationsScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libopenblas.so.0 in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner SubTypesScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libopenblas.so.0 in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner TypeAnnotationsScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libnd4j.so in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner SubTypesScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libnd4j.so in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner TypeAnnotationsScanner
13:58:59.308 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libjnind4j.so in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner SubTypesScanner
13:58:59.309 [main] DEBUG org.reflections.Reflections - could not scan file org/nd4j/nativeblas/linux-x86_64/libjnind4j.so in url jar:file:/home/fab/.m2/repository/org/nd4j/nd4j-native/0.4-rc3.10/nd4j-native-0.4-rc3.10-linux-x86_64.jar!/ with scanner TypeAnnotationsScanner
13:58:59.311 [main] INFO org.reflections.Reflections - Reflections took 212 ms to scan 6 urls, producing 90 keys and 329 values
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.deeplearning4j.examples.convolution.TraceClassifier.convertTrace(TraceClassifier.java:225)
at org.deeplearning4j.examples.convolution.TraceClassifier.createDataset(TraceClassifier.java:245)
at org.deeplearning4j.examples.convolution.TraceClassifier.main(TraceClassifier.java:80)
Caused by: java.lang.RuntimeExceptio

Exception in thread "main" java.lang.ExceptionInInitializerError
at org.deeplearning4j.examples.convolution.TraceClassifier.convertTrace(TraceClassifier.java:225)
at org.deeplearning4j.examples.convolution.TraceClassifier.createDataset(TraceClassifier.java:245)
at org.deeplearning4j.examples.convolution.TraceClassifier.main(TraceClassifier.java:80)
Caused by: java.lang.RuntimeException: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: Please ensure that you have an nd4j backend on your classpath. Please see: http://nd4j.org/getstarted.html
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:4780)
at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:146)
... 3 more
Caused by: org.nd4j.linalg.factory.Nd4jBackend$NoAvailableBackendException: Please ensure that you have an nd4j backend on your classpath. Please see: http://nd4j.org/getstarted.html
at org.nd4j.linalg.factory.Nd4jBackend.load(Nd4jBackend.java:178)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:4777)

... 4 more

BUILD FAILURE

Total time: 2.611s
Finished at: Tue Jun 28 13:58:59 CEST 2016
Final Memory: 6M/239M

I have tried both java jdk 1.7 and 1.8, no changes at all. I have also tried upgrading guava depedency to version 19... no changes. Is this a well known problem or typical error? Do you need my POM file details?
raver119
@raver119
Jun 28 2016 12:13
show your pom.xml
to avoid walls of text in chat :)
Fabiano
@ftarlao
Jun 28 2016 12:15
ah ok, I have posted here beacuse I have read this was the preferred support channel, I'll swap immediately to github issue manager. My fault
raver119
@raver119
Jun 28 2016 12:15
no
open link i've sent you
it's just a tool, to share big chunks of text
Fabiano
@ftarlao
Jun 28 2016 12:16
got it now
:-)
here is my pom:
Fabiano
@ftarlao
Jun 28 2016 12:22
Other infos: Linux Mint 17.2, current Java version: java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.14.04.1)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)
raver119
@raver119
Jun 28 2016 12:23
pretty strange.
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j</artifactId>
<version>0.4-rc2.2</version>
<type>pom</type>
what is this?
please remove that
Fabiano
@ftarlao
Jun 28 2016 12:24
I'll try immediately, this is generated by Netbeans, I have checked the file but missed it. I'll try removing it now
got the same error
I have performed a clean and build
raver119
@raver119
Jun 28 2016 12:29
show me new output please
i've never heard of such exception with cpu backend though
usually it happens only with cuda
Fabiano
@ftarlao
Jun 28 2016 12:32
I have added the compilation output of Netbeans at the previust gist
Fabiano
@ftarlao
Jun 28 2016 12:39
Days ago, with 0.4-rc3.8 I remind having the same problem that has disapperead when I updated guava to version 19.. so..I have thought being a guava bug. But now I think the problem is more subtle.
Ah, the execution output with the pom changes is exactly the same
raver119
@raver119
Jun 28 2016 12:42
show me your current pom.xml please once again, and output.
i know, they might look similar. But i need to make sure what's up there right now.
Fabiano
@ftarlao
Jun 28 2016 12:45
updated gist
raver119
@raver119
Jun 28 2016 12:45
please dont
post new one
how will i compare new vs old, if i'll not have old?
Fabiano
@ftarlao
Jun 28 2016 12:46
ok, I'll fix it, this is the first time I collaborate in this way
raver119
@raver119
Jun 28 2016 12:46
np :)
raver119
@raver119
Jun 28 2016 12:50
yea, it looks fine now. but not working :/
Fabiano
@ftarlao
Jun 28 2016 12:51
My only doubt is about Netbeans managing all the suff, and my Maven-newbie status... I am not sure what Netbeans is doing exactly. The very strange thing is that 3.8 is working.
Another strange thing is that the default dependency ..guava 11.. for dl4j 3.8 has failed to work.
and I needed to update it.
Do you think I have to try to self compile all the stuff locally and drop maven?
Do you reccomed this?
Adam Gibson
@agibsonccc
Jun 28 2016 12:53
No just use an actual ide ;)
You're in for a world of hurt if youre compiling from source
Speaking of which - unless you're contributing or doing exactly that, could you migrate to the general channel?
This channel is specifically for those kinds of questions
Fabiano
@ftarlao
Jun 28 2016 12:56
ok, I'll try another ide. I think Intellij, in case I'll post question to a more general channel.
Thankyou for support, regards, I'll update in case I found out a solution
raver119
@raver119
Jun 28 2016 12:59
yea, intellij is the best java ide atm
so use that, and you’ll be fine
Fabiano
@ftarlao
Jun 28 2016 13:08
ok thx