These are chat archives for deeplearning4j/deeplearning4j/earlyadopters

8th
Jun 2016
ianpjohnson
@ianpjohnson
Jun 08 2016 03:19
@AlexDBlack i am considering doing some flink stuff for my own project (still reading) and might take a shot at adding flink support to arbiter - would that be a problem, and can you think of anything non obvious i would have to look out for ?
Specifically, is SparkCandidateExecutor still in progress ?
Alex Black
@AlexDBlack
Jun 08 2016 03:30
yeah, spark stuff there needs a lot of work
I was going to start looking at that this week
ianpjohnson
@ianpjohnson
Jun 08 2016 03:31
@AlexDBlack I haven't looked closely yet but do you intend on using Streaming in Arbiter - many optimization algorithms would need that to direct the space search generation
Alex Black
@AlexDBlack
Jun 08 2016 03:31
so arbiter is based on hyperparameter optimization, where you are trying to optimize a certain metric (test set loss, accuracy etc)
I'm not sure how well streaming would work there
ianpjohnson
@ianpjohnson
Jun 08 2016 03:32
@AlexDBlack - Well i am new to DL4J, Spark & Flink - so don't hold your breath - i'll try and keep the questions relevant though
Well Grid & Random searh are ok but there may be better ?
Alex Black
@AlexDBlack
Jun 08 2016 03:33
better approaches than random search and grid search? yeah, we have bayesian hyperparameter optimization schemes in our closed-source extension of arbiter
similar to spearmint, if you are familiar with it
using gaussian processes
as for streaming - deploying the trained model on spark streaming or flink makes sense
ianpjohnson
@ianpjohnson
Jun 08 2016 03:34
And streaming allows feeding back into the search space depending on the prior results - i am assuming the solution you are starting with is basically batch based with termination condition - i was just looking at the README - i dont know Spermint - ill look it up
Alex Black
@AlexDBlack
Jun 08 2016 03:35
right, I mean it should be flexible enough to do other things
but the current dl4j implementation there is definitely based on fixed test sets
at the end of the day though you can in principle optimize anything that gives you a scalar score value
ianpjohnson
@ianpjohnson
Jun 08 2016 03:40
Is the the Spearmint implementation you mentioned ?
https://github.com/JasperSnoek/spearmint
Alex Black
@AlexDBlack
Jun 08 2016 03:42
yeah. there's two versions of spearmint though, and old one and a newer rewrite
both are python, based on the same ideas though
Alex Black
@AlexDBlack
Jun 08 2016 03:44
yeah, that's the two versions
ianpjohnson
@ianpjohnson
Jun 08 2016 03:45
k - i'll have a read and play with the stuff as it is
Michael Sch.
@mschaars
Jun 08 2016 09:57
@ianpjohnson you might wanna look at http://sheffieldml.github.io/GPyOpt/ too
I dont think jasper works on spearmint any more, they moved development to whetlab and he stopped responding on that after they got acquired by twitter, gpyopt is much more active
Roi Reshef
@RoiViber
Jun 08 2016 11:51
@AlexDBlack are you guys developing Spark components in scala? DL4S seems a little outdated now when importing DL4J libraries
Also, is there any doc/summary of the different modules of DL4J (e.g aws/graph/nlp/scaleout/etc.)?
Alex Black
@AlexDBlack
Jun 08 2016 12:09
no, our spark stuff is in java. And dl4s does need some work, for sure. tbh we need to hire a scala dev for that
as for modules: core, spark and ui are the main ones... they contain pretty much what you would expect
Roi Reshef
@RoiViber
Jun 08 2016 12:11
I do a little of scala+spark and have some free time. Would you like to have a talk sometime? I might give a hand on my free time.
Alex Black
@AlexDBlack
Jun 08 2016 12:17
sure, feel free to direct message any time... we'd certainly appreciate contributions there
raver119
@raver119
Jun 08 2016 14:47
@treo i have issues on current libnd4j master with cuda
[ERROR] Unknown opNum=5 on C:/Users/raver/develop/libnd4j/include\transform.h:253
and there's lots of such messages
[ERROR] Unknown opNum=16 on C:/Users/raver/develop/libnd4j/include\pairwise_transform.h:91
have you run tests before merge?
Paul Dubs
@treo
Jun 08 2016 14:52
Does cuda use different opnums?
Also, no shame on me.
I just made sure everything compiled
raver119
@raver119
Jun 08 2016 14:56
no, it's the same opnums
Paul Dubs
@treo
Jun 08 2016 14:56
then I must have messed something up
raver119
@raver119
Jun 08 2016 14:56
heh, good we hadn't merged before release :)
Paul Dubs
@treo
Jun 08 2016 14:57
:D
raver119
@raver119
Jun 08 2016 14:57
code freeze is magic!
Paul Dubs
@treo
Jun 08 2016 14:59
I just don't get how it can happen that all of it falls through to the else
raver119
@raver119
Jun 08 2016 15:05
where's DISPATCH_BY_OPNUM defined?
i can't find it
raver119
@raver119
Jun 08 2016 15:05
lol
ok thx
usual suspect is MSVC
raver119
@raver119
Jun 08 2016 15:16
@treo any other ideas?
Paul Dubs
@treo
Jun 08 2016 15:17
Still trying to get msvc to simple spit out how it expands that macro
raver119
@raver119
Jun 08 2016 15:18
or nvcc...
last answer
Paul Dubs
@treo
Jun 08 2016 15:20
but it doesn't complain about the macros, or does it?
and that answer is over 8 years old
raver119
@raver119
Jun 08 2016 15:21
yea, there's no such key
and no, it doesn't complain or anything.
it just sees no ops
raver119
@raver119
Jun 08 2016 15:27
as far as i understand, since we have error messages - it works
error messages are coming from device functions
so it was processed
Paul Dubs
@treo
Jun 08 2016 15:29
It looks like MSVC is to blame, as it stops processing the macro after the first step for some reason
Paul Dubs
@treo
Jun 08 2016 15:38
ok, the reason seems to be that MSVC doesn't fully support __VA_ARGS__ yet, but it looks like a solution is possible
raver119
@raver119
Jun 08 2016 15:41
cool, waiting for your commit then
Paul Dubs
@treo
Jun 08 2016 15:48
pushed it
raver119
@raver119
Jun 08 2016 16:05
fix confirmed
Paul Dubs
@treo
Jun 08 2016 16:05
:+1:
let's hope we will never have to touch that macro again
raver119
@raver119
Jun 08 2016 16:06
lol
Paul Dubs
@treo
Jun 08 2016 16:06
its voodoo content is way to high, but it removed 1k lines of boilerplate
Adam Gibson
@agibsonccc
Jun 08 2016 16:09
Lol
Paul Dubs
@treo
Jun 08 2016 16:11
and not just any boilerplate, but a rainforest kind of paradise for bugs to hide in... so it is a somewhat necessary evil
Adam Gibson
@agibsonccc
Jun 08 2016 16:17
Haha
raver119
@raver119
Jun 08 2016 16:19
damn, you're not the only one with bugged commit...
my branches are broken too :(
god bless code freeze
strange random bug with scalars and transforms
Paul Dubs
@treo
Jun 08 2016 16:20
yep, code freeze is very necessary for at least somewhat stable releases