These are chat archives for deeplearning4j/deeplearning4j/earlyadopters

18th
Jun 2016
vjanand
@vjanand
Jun 18 2016 17:04

Here is the situation - I have two different files (one for training) and the other for testing. Here is the code sample:

//This is for creating the training set
File trainingFile = new File("/mnist_train_100.csv");
RecordReader reader = new CSVRecordReader(numberofLinesToSkip, delimitter);
reader.initialize(new FileSplit(trainingFile));
DataSetIterator iter = new RecordReaderDataSetIterator(reader, batchSize, lableindex, possibleClass);

//This is for creating the testing set
File testingFile = new File ("mnist_test_10.csv");
RecordReader readerTest = new CSVRecordReader(numberofLinesToSkip, delimitter);
reader.initialize (new FileSplit(testingFile));
DataSetIterator iterTest = new RecordReaderDataSetIterator(readerTest,1,lableindex,possibleClass);

I am able to get the training set read and the model gets trained. But, I am unable to execute the testing part, as the size shows as zero and the testing loop never gets executed. What am I doing wrong?

Adam Gibson
@agibsonccc
Jun 18 2016 17:06
@vjanand Have you thought about stepping through with the debugger?
A lot of the times with something like that its usually a simple mistake
vjanand
@vjanand
Jun 18 2016 17:07
Sure - but you don't see any obvious mistake in my usage, correct?
Adam Gibson
@agibsonccc
Jun 18 2016 17:07
Not really no
Train/test iterators are pretty obvious
Is your batch size really 1 though?
vjanand
@vjanand
Jun 18 2016 17:09
I have about 10 examples to test..
So, I am creating a batch size of 1?
Or should I just make all the 10 as a batch size?
Adam Gibson
@agibsonccc
Jun 18 2016 17:09
Look closely at your constructors in the 2 iterators
I don't see why you wouldn't
It's not like 10 is that big
vjanand
@vjanand
Jun 18 2016 17:10
yes -- but, I was just trying out; Ok I will make the whole file as one batch
vjanand
@vjanand
Jun 18 2016 17:16
Thanks Adam -- as you suggested a simple error
Adam Gibson
@agibsonccc
Jun 18 2016 17:16
Cool :D
Yeah it often is
vjanand
@vjanand
Jun 18 2016 18:23
A question on backpropagation implementation - is there an explicit setting on the NN to choose between online vs batch updating? Or batchsize implicitly defines that (a batch size of 1 defaults to online)?
Adam Gibson
@agibsonccc
Jun 18 2016 18:23
@vjanand Please move those kind of questions to tuning help: https://gitter.im/deeplearning4j/deeplearning4j/tuninghelp
This channel is more for people having issues building from source and the like
vjanand
@vjanand
Jun 18 2016 18:35
ok, will do
Drew Shaw
@ds923y
Jun 18 2016 20:09
Are there one or 2 function calls for Nd4j that will do an epsilon equals where the epsilon is specified. Right now I create two matrices with epsilon added and subtracted to matrix A. Then call AplusEpsilon.gt(B) and AminusEpsilon.lt(B) with these I find the .minNumber and .maxNumber of both of these and see if all these resulting java.lang.Number's equal 1