These are chat archives for deeplearning4j/deeplearning4j

25th
Sep 2018
C4N4D4M4N
@C4N4D4M4N
Sep 25 2018 03:12
Does DL4J support simulated annealing?
Alex Black
@AlexDBlack
Sep 25 2018 03:32
as a training/optimization method? no
I'm not aware of it being used as a practical tool in any of the deep learning libraries
Rhicheek Patra
@rpatra
Sep 25 2018 07:55
@AlexDBlack It seems there are two distributed word2vec:
org.deeplearning4j.spark.models.embeddings.word2vec in dl4j-spark-nlp_2.11, and
org.deeplearning4j.spark.models.word2vec.SparkWord2Vec in dl4j-spark-nlp-java8_2.11
Which one should be used?
Alex Black
@AlexDBlack
Sep 25 2018 08:16
the distributed word2vec is basically for very large vocabularies - larger than you can fit on a single node
it does add some overheads, so don't use spark unless you have to
that said, I think you want the former (cc @raver119)
liuxun666
@liuxun666
Sep 25 2018 08:27
what's mean "Warning: 1 class was never predicted by the model and was excluded from average precision"?
gitterBot
@raver120
Sep 25 2018 08:27
@liuxun666 Welcome! Here's a link to Deeplearning4j's Gitter Guidelines, our documentation and other DeepLearning resources online. Please explore these and enjoy! https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j/GITTER_GUIDELINES.md
Alex Black
@AlexDBlack
Sep 25 2018 08:30
@liuxun666 It means exactly what the warning says
your model never predicted class 1 for some reason
the precision for class 1 is therefore undefined, and hence it's excluded from the average
liuxun666
@liuxun666
Sep 25 2018 08:33
@AlexDBlack Every class exists in the data. Where should I start to solve this problem?
Alex Black
@AlexDBlack
Sep 25 2018 08:34
see: !tuning
it's often due to the network not learning properly due to bad tuning - i.e., bad learning rate, or no data normalization, for example
tranva
@tranva
Sep 25 2018 08:35
or data imbalance
Alex Black
@AlexDBlack
Sep 25 2018 08:35
right, or that
there's many possible causes unfortunately
liuxun666
@liuxun666
Sep 25 2018 08:36
tks
MK
@mckunkel
Sep 25 2018 08:36

@AlexDBlack Greetings, again, so I have made an ObjectRecordReader for my project. Everything seems to be in order. I construct my model with the output layer being a Yolo2OutputLayer as in the example HouseNumberDetection.java. However when I run my model I get the following error, which I do not see when running the example HouseNumberDetection.java

Exception in thread "main" org.nd4j.linalg.exception.ND4JIllegalStateException: Shape of target array for reduction [[1, 53, 1]] doesn't match expected [[1, 112, 6]]

Since I do not see this error in the example, I assumed it was my record reader, but I find not problems there.
Any insights for me?

Alex Black
@AlexDBlack
Sep 25 2018 08:37
@mckunkel mind posting the full stack trace in a !gist?
gitterBot
@raver120
Sep 25 2018 08:37
To use gist: paste your code/exception/large output log into https://gist.github.com, click 'Create Secret Gist' and paste URL link here
MK
@mckunkel
Sep 25 2018 08:39
@AlexDBlack Thanks, the stacktrace can be found
https://gist.github.com/mckunkel/06fcc34ec5455f19ac56150f6d4cbc3b
Alex Black
@AlexDBlack
Sep 25 2018 08:41
so first guess is that the spatial dimensions of your output don't match the spatial dimensions of the labels
you can use that method to inspect the activation shapes for each layer
MK
@mckunkel
Sep 25 2018 08:43
OK, thanks, I will have a look now
Ahh, yes the summary. I did that
and I saw nothing, let me post that to gist
@AlexDBlack I updated the gist
https://gist.github.com/mckunkel/06fcc34ec5455f19ac56150f6d4cbc3b
to reflect this summary.
Alex Black
@AlexDBlack
Sep 25 2018 08:46
ok, mind posting your ObjectDetectionRecordReader config too?
Alex Black
@AlexDBlack
Sep 25 2018 08:55
so, your network downsamples from 112x6 to 53x1
yet your data pipeline assumes the output size is the same as the input
so either fix your data pipeline (i.e., remove that assumption)
or fix your network (so there's no downsampling - i.e., outSize == inSize)
MK
@mckunkel
Sep 25 2018 08:57
In the example HouseNumberDetection.java, where are either of these assumptions handled?
MK
@mckunkel
Sep 25 2018 08:59
Because I do not see where it is set in my pipeline where the first can be changed, and if I changed the network, then this doesnt perform as I want it
Alex Black
@AlexDBlack
Sep 25 2018 08:59
it's possible to configure your net to have output size == input size
which might be reasonable in your case
it's basically a case of stride 1, no pooling layers, and ConvolutionMode.Same
MK
@mckunkel
Sep 25 2018 09:02
Cool, I will try that idea now.
I see what you mean in the HouseNumberDetection.java. Are the locations of the objects in the example relative to the final grid of h=13 w = 13, or to the pixel location of the image itself?
because I think I do not understand this from the papers
Alex Black
@AlexDBlack
Sep 25 2018 09:04
there's a correspondence between the input and output
each of the 13x13 output "pixels" represents a block of 32x32 input pixels
each object is present in one of them
and then there's an offset - i.e., a position within the output pixel
that's part of what the net predicts
0 would mean left side, 1 means right side in that case
MK
@mckunkel
Sep 25 2018 09:09
I understand, so in my case where I have an input of 112x6 "pixels", I would have to define a gridbox in which I think one of the objects is located
for your first suggestion, if I may pic your brain a bit more today. Your idea of a network with input = output. Was it a basic 1 layer convolution layer connected to a yolooutput? or did I misunderstand?
Alex Black
@AlexDBlack
Sep 25 2018 09:11
it'd have multiple layers, otherwise each output pixel would only have a very small region upon which to make decisions
if you go for downsampling though, I think your only option would be 2x downsampling, so 56x3 output; each pixel corresponding to a 2x2 input pixels
not sure how well that would work in practice though
MK
@mckunkel
Sep 25 2018 09:16
so i would have to include padding to the Convolution layers also, because a stride of 1 reduces the size also
Alex Black
@AlexDBlack
Sep 25 2018 09:23
that's what ConvolutionMode.Same is for
MK
@mckunkel
Sep 25 2018 09:25

What is a 2x downsample? 2x2 kernel with stride 2x2?

if you go for downsampling though, I think your only option would be 2x downsampling

Alex Black
@AlexDBlack
Sep 25 2018 09:28
inputSize/outputSize = 2
MK
@mckunkel
Sep 25 2018 09:28
@AlexDBlack nice, thanks, I didnt see that that trick in the book
MK
@mckunkel
Sep 25 2018 09:35

@AlexDBlack So going with the fix of the pipeline I now get a different reshape error, that I think has to to with the Yolo Layer itself reshaping from

Input format: [minibatch, B*(5+C), H, W] -> Reshape to [minibatch, B, 5+C, H,
 * W

new error is

New shape length doesn't match original length: [15960] vs [4200]. Original shape: [1, 25, 56, 3] New Shape: [1, 5, 19, 56, 3]
nOut of the last conv layer must be b*(5+c)
MK
@mckunkel
Sep 25 2018 09:38
It is
Alex Black
@AlexDBlack
Sep 25 2018 09:40
ok, I need the full stack trace... and the number of classes
MK
@mckunkel
Sep 25 2018 09:47
Thanks, I have the full stack trace at
https://gist.github.com/mckunkel/06fcc34ec5455f19ac56150f6d4cbc3b#file-stacktraceforobject
While the number of classes is set to 14
Alex Black
@AlexDBlack
Sep 25 2018 09:53
that's a completely different stack trace to the error posted above...
MK
@mckunkel
Sep 25 2018 09:53
@AlexDBlack I think now I have a different problem with the new pipeline settings, so please allow me to debug my RecordReader.
Yeah, I ran it again and noticed it was different
It looks like its coming from the RecordReader, I will have a debugging session now
Alex Black
@AlexDBlack
Sep 25 2018 09:56
anyway, the expected input and label shapes are listed in the layer javadoc
make sure they are right
it's either input size, label size, or the priors array size that are wrong/mismatched
MK
@mckunkel
Sep 25 2018 09:57
@AlexDBlack Thanks
陈焕生
@yelangchs
Sep 25 2018 10:42
@AlexDBlack
// i.e. [1, nan, 3, nan]
INDArray a = ND4j.create(...)
INDArray out = Nd4j.getExecutioner().execAndReturn(new MatchConditionTransform(d1, Conditions.isNan()));
out = Transforms.not(out);
// I want get only True by "out" index.
so the result is [1, 3], and how to get it?
[1, nan, 3, nan]
out = [1, 0, 0, 1]
result = [1, 3]
Igor Markiewicz
@imarkiew
Sep 25 2018 10:52
@saudet If I understand it well, it requires Schema. If I have records with a lot of attributes, generating this manually would be exhausting. This is the only option to remove the column ?
Alex Black
@AlexDBlack
Sep 25 2018 11:35
@imarkiew generating a schema for your purpose should be a few lines. You can batch add columns according to a pattern (there's methods on the schema builder for that)
Samuel Audet
@saudet
Sep 25 2018 12:41
@imarkiew another option would be to subclass the record reader, it's not so hard, see a simple example here:
Alex Black
@AlexDBlack
Sep 25 2018 13:06
@yelangchs so yes that is possible, but unfortunately there was a bug in 1.0.0-beta2 regarding the op you would need to do it
on !snapshots (see below) you can use Nd4j.where(out, null, null) to get the result array you want
https://github.com/deeplearning4j/deeplearning4j/blob/c2c349197c2bff4a1f1e8b8d5db832750616cdbb/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/factory/Nd4j.java#L6003-L6015
gitterBot
@raver120
Sep 25 2018 13:06
Please check out this guide to set up snapshots in your project: https://deeplearning4j.org/docs/latest/deeplearning4j-config-snapshots
Igor Markiewicz
@imarkiew
Sep 25 2018 13:27
@AlexDBlack I suppose I should use the getLabels method. Unfortunately JDBCRecordReader does not support this yet.
Alex Black
@AlexDBlack
Sep 25 2018 13:35
getLabels is for the label classes if you are doing classification
if all you want to do is remove a column, you don't need it
just use one of the addColumnsX methods, add your columns, remove the one you want
https://github.com/deeplearning4j/deeplearning4j/blob/1b984dd41d484509dd5274990e20b8a40933aacb/datavec/datavec-api/src/main/java/org/datavec/api/transform/schema/Schema.java#L727-L762
or do it the way Samuel suggested, subclass the record reader and override the next method to remove by index
Igor Markiewicz
@imarkiew
Sep 25 2018 13:58
@AlexDBlack Yes, but that's what I mean. The addColumnsX method gets column names and I want to get them from JDBCRecordReader. I do not see the method in JDBCRecordReader that would allow this.
Jiří Vahala
@snurkabill
Sep 25 2018 14:43
@saudet could you, please, point me to the file where are proper DataIterators used for RL?
C4N4D4M4N
@C4N4D4M4N
Sep 25 2018 16:42
is it possible to change the model's output layer loss function between epochs? So I could do MAE loss for a while, then when that yields less results, I can swap to MSE to get the model to better fit the extreme data points
Caleb
@gordoncaleb
Sep 25 2018 18:00
Hi guys, has anyone had a chance to look at deeplearning4j/deeplearning4j#6364 ? There's some sort of broadcast shape check in ND4j 1.0.0-beta2 that doesn't seem right. Or maybe autobroadcasting isn't supported and supposed to be done some other way? Thanks!
gitterBot
@raver120
Sep 25 2018 18:00
@gordoncaleb Welcome! Here's a link to Deeplearning4j's Gitter Guidelines, our documentation and other DeepLearning resources online. Please explore these and enjoy! https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j/GITTER_GUIDELINES.md
katherynliza
@katherynliza
Sep 25 2018 21:04
Hello
gitterBot
@raver120
Sep 25 2018 21:04
@katherynliza Welcome! Here's a link to Deeplearning4j's Gitter Guidelines, our documentation and other DeepLearning resources online. Please explore these and enjoy! https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j/GITTER_GUIDELINES.md
Samuel Audet
@saudet
Sep 25 2018 21:37
@snurkabill they are not used, it goes through subclasses of MDP
Jiří Vahala
@snurkabill
Sep 25 2018 21:52
@saudet thank you, I was probably looking for something else. Will check it one more time
@saudet do you have any idea if in RL4J can be easilly implemented AlphaGo?
Samuel Audet
@saudet
Sep 25 2018 22:51
that's not really RL, but IIRC there's an implementation of it for DL4J somewhere
João Batista
@ioannisbaptista
Sep 25 2018 23:05
Hello guys! Could anyone please help me with issue #6472? :pray: I'm really stuck :cry: