Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Feb 24 2018 05:26
    @agibsonccc banned @harsh1795
  • Apr 18 2017 22:39
    @agibsonccc banned @hbb21st
  • Feb 28 2017 21:37
    @raver119 banned @dims12
  • Nov 28 2016 13:15
    User @agibsonccc unbanned @cloakedstudios_twitter
  • Nov 28 2016 13:02
    @agibsonccc banned @cloakedstudios_twitter
  • Jul 20 2016 10:38
    @agibsonccc banned @404430928
  • Jun 30 2016 06:54
    @agibsonccc banned @kumarVishwesh
raguenets
@montardon
I'm using zoo Unet model with well known image set. How comes I can not train it easily ? I tried many combinations for normalization (the usual ones). My learning rate is small , but smaller is under float precision. I'm stuck and do not know how to go on.
s1nned
@s1nned
Hi @AlexDBlack, here are the informations about the strange cycles you asked for: no. of examples 20370
minibatchsize 70
grafik.png
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder() .seed(320) .weightInit(WeightInit.XAVIER) .updater(new Adam(1e-2)) .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue) .gradientNormalizationThreshold(0.5) .list() .layer(new LSTM.Builder() .activation(Activation.TANH) .nIn(4) .nOut(8) .build()) .layer(new RnnOutputLayer.Builder(LossFunctions.LossFunction.MCXENT) .activation(Activation.SOFTMAX) .nIn(8) .nOut(numLabelClasses) .build()) .build();
s1nned
@s1nned
the data is sequential (that's why i used rnn config)
Alex Black
@AlexDBlack
hm... learning rate is a little high (average of around -3 on the bottom left chart would be ideal)
configuration looks fine (other than being a little small, you could go larger with that number of examples I think)
otherwise - only guess is there's some sort of regularity in the examples? like every second minibatch is alternatively longer/shorter than the previous
or non-shuffled data or something... that's really the only idea I have right now
s1nned
@s1nned
with "average of around -3 on the bottom left chart.." you mean to set the lr to 1e-3 right?
go larger with more layers or just no. of hidden nodes?
s1nned
@s1nned
is it absolutely necessary to shuffle the data? because i know that some of the examples are labeled over a certain period of time (start time - end time) and when i shuffle the data, its scattered and the algo does not recognize any dependencies?
grafik.png
this pic shows 2500 epochs (previous was 200)
raguenets
@montardon
Hi, any hints to train my model ? I think the question is specific to DL4J. I gave code, link to image samples. I read the book, trained on Coursera . Everything required to ask question about this new to me library.
raver119
@raver119
answer is trivial
DL is still an art
raguenets
@montardon
image.png
@raver119 Here the artist.
siddadel
@siddadel
@AlexDBlack I have 15,000 images each 104 x 132 that I am training using a similar neural network as the DL4J example, org.deeplearning4j.examples.convolution. AnimalClassification. I tried both AlexNet and LeNet batch size 128, 100 epochs, but in both instances the score plateaus at 1.5. The accuracy is limited to 33-37%.
Can you provide some advice and some literature that I could read and use the UIServer graphs to tune/re-architecture my network?
siddadel
@siddadel
Forgot to add "please" ^^ :)
siddadel
@siddadel
Thanks @saudet .
Alex Black
@AlexDBlack
@siddadel hm... there's a few bad design choices there
like you first increase then decrease depth as you go through the net? it should increase as spatial dimensions decrease (so that activations h x w x depth is about constant through the net)
your achitecture seems to be based on alexnet... which is old at this point
cut the LRN (it's an obsolete technique), reduce the kernels to 3x3 (or 5x5 at most). If you need to stabilize activations, use batch norm instead
weight init - just use xavier, not manual distributions like that
cut the gradient norm completely
unless your dataset is huge, those fuly connected layers are too large
I don't know how well you have tuned the learning rate either, definitely check that in the ui
siddadel
@siddadel
@AlexDBlack - This is really helpful. I am new to neural networks so I haven't done much tuning. This will be a good start. Will go through the literature linked above and your comments and get back.
s1nned
@s1nned
with "average of around -3 on the bottom left chart.." you mean to set the lr to 1e-3 right?
go larger with more layers or just no. of hidden nodes?
is it absolutely necessary to shuffle the data? because i know that some of the examples are labeled over a certain period of time (start time - end time) and when i shuffle the data, its scattered and the algo does not recognize any dependencies? @AlexDBlack
Alex Black
@AlexDBlack

with "average of around -3 on the bottom left chart.." you mean to set the lr to 1e-3 right?

no. I mean look at the chart. If it's higher than -3 on average, decrease the learning rate. If it's lower, increase it. That's usually a good starting point

go larger with more layers or just no. of hidden nodes?

both. For that number of examples, I'd probably bump it up to something like 2x LSTM layers of size 32 or 64

is it absolutely necessary to shuffle the data?

no. It can sometimes add a percent or two to accuracy vs. not shuffling, but it's not critical
obviously for time series you just want to change the order in which those series are presented to the net, not shuffle within the time steps or anything

gharick
@gharick
hello guys, please a simple question, I have a trained network which is ready and loaded to use it for prediction
tryed output, feedforward , predit and i get an exception that tells expecting rank(2) array and recieved rank 1 array !
gharick
@gharick
figured out
LiNKeR
@54LiNKeR
Hi, what should my prior boxes be if I am training a YOLO2 from sctratch?
Samuel Audet
@saudet
@54LiNKeR something close to the typical sizes of your objects
Ramid Khan
@ramidzkh
(I'm new to DL4J) How can I train a network which plays a simple game and learns through self-play?
gitterBot
@raver120
@ramidzkh Welcome! Here's a link to Deeplearning4j's Gitter Guidelines, our documentation and other DeepLearning resources online. Please explore these and enjoy! https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j/GITTER_GUIDELINES.md
Shixuebin
@Shixuebin
Now I'm doing the detection task. When the category is 5, the effect is OK. But there are two categories that are often predicted to be the same category. What's the reason? I use tiny yolov3
Samuel Audet
@saudet
achilep
@achilep
hello my name is achile. i am newbie in deeplearning4J .can someone help me with the resource to learn ?a mentor will be welcome.thank you
gitterBot
@raver120
@achilep Welcome! Here's a link to Deeplearning4j's Gitter Guidelines, our documentation and other DeepLearning resources online. Please explore these and enjoy! https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j/GITTER_GUIDELINES.md
f
@SidneyLann
image.png
I'm training a sd nn, why there is only one cpu core has sy time? why the cpu cores which have no sy time not use most the time ie 95%+?
f
@SidneyLann
image.png
beta6 solved this issue
f
@SidneyLann
This issue still exists in beta6 when using mkl_rt
f
@SidneyLann
There is no io waiting time, why openblas can use the most cpu time but mkl can't? @saudet
Samuel Audet
@saudet
MKL is better at deciding when using more threads would actually increase processing time. That's probably what is happening here. In any case, we can usually set OMP_NUM_THREADS to adjust that manually.
domkat bess
@domkatbess
Hello Everyone, itz nice to be here. I am new to deep learning, so i got here through deeplearning4j.org
gitterBot
@raver120
@domkatbess Welcome! Here's a link to Deeplearning4j's Gitter Guidelines, our documentation and other DeepLearning resources online. Please explore these and enjoy! https://github.com/deeplearning4j/deeplearning4j/blob/master/deeplearning4j/GITTER_GUIDELINES.md
domkat bess
@domkatbess
thanks @raver120
gitterBot
@raver120
Hm, maybe we should upgrade me to something that involves some magic AI?
f
@SidneyLann
image.png
f
@SidneyLann
loss not decrease after 50+ epoches and now 800+ epoches. tune learning rate or mini batch size no help, what the problem should be?