These are chat archives for deeplearning4j/deeplearning4j/earlyadopters

9th
Nov 2017
Steven Lang
@steven-lang
Nov 09 2017 02:51
Hey, I've just setup everything accoding to the building locally section (CPU, openblas). Somehow the maven installation fails at some point (maven shade plugin) with the following message: gist. Anyone any clue how this can be solved?
Adam Gibson
@agibsonccc
Nov 09 2017 02:52
@slang03 can you give us a whole build log?
Steven Lang
@steven-lang
Nov 09 2017 02:53
Yes, its quite long, i'll just need to rerun and store it
Are you somewhere forcing to redownload all maven dependencies in the build script?
Adam Gibson
@agibsonccc
Nov 09 2017 02:56
that shade error doesn't make any sense to me
that's why I'm asking for a more complete build log
Steven Lang
@steven-lang
Nov 09 2017 02:57
Yeah it's currently running. Just takes some time for the dependency downloads
Was just a side question
Adam Gibson
@agibsonccc
Nov 09 2017 02:57
oh
you want an offline mode
I wonder if it's downloading out of date snapshots
Steven Lang
@steven-lang
Nov 09 2017 02:57
I was suprised to see it downloading everything again even though I just ran the script a few minutes before
Adam Gibson
@agibsonccc
Nov 09 2017 02:58
what is it downloading though?
is it the dl4j deps?
I only need nd4j re run..it shouldn't be redownloading anything
Steven Lang
@steven-lang
Nov 09 2017 02:59
I've just restarted the whole stack build script
Adam Gibson
@agibsonccc
Nov 09 2017 03:00
k
Steven Lang
@steven-lang
Nov 09 2017 03:09
Funny, the issues did not appear on the second run
classic demo effect
Jakub Peschel
@papperwing
Nov 09 2017 09:17
I ran the build-dl4j-stack and it didn't build nd4j cuda and eveerything after that, because libnd4j with cuda support wasnt build first. So I tried to follow https://github.com/deeplearning4j/libnd4j/blob/master/windows.md and I installed Visual Studio 2015, but there is not any batch file called vcvars64.bat to set environment. In case of newer Visual Studio there is even no bin directory in VC folder. Is there way how to build it without that batch file?
Alex Black
@AlexDBlack
Nov 09 2017 09:19
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\vcvars64.bat
that's where it should be
Jakub Peschel
@papperwing
Nov 09 2017 09:19
It is not there
I search whole folder for it
Alex Black
@AlexDBlack
Nov 09 2017 09:19
did you install support for c++?
that's really the only thing I can think of here
I mean I installed VS 2015 from the ISO on a PC not that long ago...
en_visual_studio_community_2015_x86_dvd_6847364.iso specifically
Jakub Peschel
@papperwing
Nov 09 2017 09:21
I used en_visual_studio_community_2015_with_update_3_x86_x64_web_installer_8922963.exe
And it installed vs without any setting from me
I will check it
Alex Black
@AlexDBlack
Nov 09 2017 09:22
yeah, I haven't used the web installer. But I know the ISO gives you the option to install a bunch of different modules
Jakub Peschel
@papperwing
Nov 09 2017 09:23
The 2017 version did but 2017 dont have support for cuda 8 which I want to use
Jakub Peschel
@papperwing
Nov 09 2017 09:50
@AlexDBlack It was the issue after figuring out how to install c++ stuff it is there. Thank you
Alex Black
@AlexDBlack
Nov 09 2017 09:51
sure, np
Jakub Peschel
@papperwing
Nov 09 2017 09:56
Ok it still didn't help I hoped that problem with path will be solved after setting env vars.
https://gist.github.com/papperwing/7b4a7b4c852899c248a305527599b467
I tried to run vcvarsall.bat
but it didnt help
Alex Black
@AlexDBlack
Nov 09 2017 10:08
try opening cmd window, running vcvars64, then "c:\msys64\mingw64.exe" then try the build in msys console
Jakub Peschel
@papperwing
Nov 09 2017 10:09
that is exactly what i did
Alex Black
@AlexDBlack
Nov 09 2017 10:12
and you have MSYS2_PATH_TYPE=inherit in mingw64.ini?
Jakub Peschel
@papperwing
Nov 09 2017 10:12
Yes I do
I tested it on maven which wouldnt be running without it
I guess
I removed # from that ini file
Alex Black
@AlexDBlack
Nov 09 2017 10:13
what does "where cl" return in mingw?
Jakub Peschel
@papperwing
Nov 09 2017 10:14
$ where cl
INFO: Could not find files for the given pattern(s).
Alex Black
@AlexDBlack
Nov 09 2017 10:14
well, there's your problem
c:>where cl
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\cl.exe
Jakub Peschel
@papperwing
Nov 09 2017 10:15
Yeah I have it there bud mingw is not seeing that
Alex Black
@AlexDBlack
Nov 09 2017 10:15
right, because it's not on your path
Jakub Peschel
@papperwing
Nov 09 2017 10:15
How does your mingw.ini looks? does it one commant at line or is it all in one line?
Alex Black
@AlexDBlack
Nov 09 2017 10:15
add C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64 to PATH env variable
I only modified that one line
Jakub Peschel
@papperwing
Nov 09 2017 10:16
I have whole .ini file in one line
which is bit confusing for me as linux user :D
Jakub Peschel
@papperwing
Nov 09 2017 10:52
Ok I built native operations for cuda in libnd4j then I started build-dl4j-stack and it failed again on error that i need to build libnd4j with cuda support first. Could you guys give me some advice how to solve it?
Alex Black
@AlexDBlack
Nov 09 2017 10:56
once the build is complete (successfully) really the only thing that can cause it is LIBND4J_HOME not being set:
https://github.com/deeplearning4j/libnd4j/blob/master/windows.md#building-nd4j
Jakub Peschel
@papperwing
Nov 09 2017 10:57
Ok so I will build native operations for cuda then set LIBND4J_HOME and then call ./build-dl4j-stack.sh?
I will try that :)
Jakub Peschel
@papperwing
Nov 09 2017 11:23
Ok when I run directly mingw64.exe it did not pass path to that shell but when i run C:\msys64\msys2_shell.cmd -mingw64 it does. In both configs I have same stuff uncommented and I also added that line (MSYS2_PATH_TYPE=inherit) into environment variables. Did you meet this behavior? I am not able to properly run buildnativeoperations.sh -c cuda -cc 61 because of that and I am not able to build nd4j-cuda because of missing built libnd4j with cuda support.
And I was mistaken that I build operations correctly
I accidentally run just ./buildnatveoperations.sh without parameters for cuda
Snímek obrazovky (18).png
ok I maybe did progress by starting mingw64 with mentioned command from cmd where i run vcvars64.bat
But I am starting to really hate Windows more than ever :D Everything is so complicated in them :D
Jakub Peschel
@papperwing
Nov 09 2017 12:27
Did I forget to download something?
Paul Dubs
@treo
Nov 09 2017 13:08
I haven't built locally in a while, but maybe you have to explicitly set the scala version?
Adam Gibson
@agibsonccc
Nov 09 2017 13:10
@treo you generally want to use the buildmultiplescalaversions.sh script in dl4j
Jakub Peschel
@papperwing
Nov 09 2017 13:12
It was meant as answer to my question. Do I want to run buildmultiplescalaversions.sh before build-dl4j-stack.sh?
Adam Gibson
@agibsonccc
Nov 09 2017 13:12
no..build-dl4j-stack.sh takes params
either use it as is or invoke these scripts manually
otherwise you might get unintended side effects
Jakub Peschel
@papperwing
Nov 09 2017 13:13
ok
still I think that problem is somewhere else, because I have seen in build log that scala modules were build somewhere there.
Adam Gibson
@agibsonccc
Nov 09 2017 13:15
oh weird
 [ERROR] Could not find the selected project in the reactor: dl4j-examples/tutorials/docker @
[ERROR] Could not find the selected project in the reactor: dl4j-examples/tutorials/docker -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
if I had to guess, that might be related to @crockpotveggies in some form
I'd check with him on the docker related stuff
Jakub Peschel
@papperwing
Nov 09 2017 13:16
Ok Thank you :)
Adam Gibson
@agibsonccc
Nov 09 2017 13:16
@papperwing so please try to dig in to this a bit
I'm providing you my read of this just looking at it..I've been building dl4j fine
Looking at this the dl4j parent pom doesn't have this project even in there: https://github.com/deeplearning4j/deeplearning4j/blob/master/pom.xml#L493
so this shouldn't be a build "failure" I don't think
your "error" here seems pretty mysterious
I don't run the build script myself
I just use the per project build scripts and update parts as needed
that's how most of us do it
That's also why I suggested to just run the individual build scripts
in general, once you have nd4j built the rest is pure java
Jakub Peschel
@papperwing
Nov 09 2017 13:19
Ok
Jakub Peschel
@papperwing
Nov 09 2017 13:36
I just pulled and I am trying tu build it one more time, because there are some changes in docker example in master. Maybe I just was lucky enough to pick the repo in bad moment :)
Adam Gibson
@agibsonccc
Nov 09 2017 13:36
the docker build isn't tied to dl4j fyi
that's why I was telling you to run the dl4j repo buildmultiplescalaversions.sh build by itself
I showed you that earlier
there was even a pom link
if it was my answer would have been different
did you get nd4j built?
Jakub Peschel
@papperwing
Nov 09 2017 13:38
Yeah I understand that I am looking into script and trying to figure stuff out :)
Adam Gibson
@agibsonccc
Nov 09 2017 13:38
what about libnd4j?
if both of those are built the individual scripts should be enough
then if you get an error post that
Jakub Peschel
@papperwing
Nov 09 2017 13:38
ok
Jakub Peschel
@papperwing
Nov 09 2017 14:47
Ok just one quick info sideway, I ran build after pulling and it was successfully built. I will search where the project dl4j-examples got into building because it is not part of pom.xml as you mentioned but I wanted to let you know that Adam.
Justin Long
@crockpotveggies
Nov 09 2017 15:47
@agibsonccc @papperwing that's been fixed yesterday please pull latest changes
Jakub Peschel
@papperwing
Nov 09 2017 15:48
Yeah I figured that out :) Thank you :)
Steven Lang
@steven-lang
Nov 09 2017 21:41
Is it possible to contribute to the documentation at https://deeplearning4j.org ?
Alex Black
@AlexDBlack
Nov 09 2017 22:32
basically just a PR to gh-pages branch
Steven Lang
@steven-lang
Nov 09 2017 22:33
oh okay, i've searched in master but missed the classic gh-pages branch. thanks!
Alex Black
@AlexDBlack
Nov 09 2017 22:49
@akhodakivskiy the most likely reason for nd4j complaining is that you don't have LIBND4J_HOME environment variable set
assuming you're on linux, run export LIBND4J_HOME=/path/to/libnd4j/ then try the nd4j build again