These are chat archives for beniz/deepdetect

2nd
Nov 2016
Kumar Shubham
@kyrs
Nov 02 2016 21:00
hi @beniz I am trying to train dd for Agnews data using character based input and with embed layer. As per our previous discussion I have already merged beniz/deepdetect#174 into my local branch. Do you think the .prototxt files in https://github.com/beniz/deepdetect/pull/189/files will work for my case ??
Emmanuel Benazera
@beniz
Nov 02 2016 21:02
hello @kyrs this is because you need an embedding before the lstm layer ?
Kumar Shubham
@kyrs
Nov 02 2016 21:04
yes ... before testing the changes I have made in the code . I want to validate if I have a proper prototxt file to do so.
Emmanuel Benazera
@beniz
Nov 02 2016 21:07
I think you should be able to modify these files to keep the embedding and add the lstm layer on top... you'll need to pass the deltas from the input straight to the lstm though I believe
there was an example in one of the Caffe PRs
this one is a bit more complicated because it's for image captioning but you can see the lstm layer and its inputs. The embedding here is produced by a CNN but you can get the same type of tensors from text with a simple embedding layer.
Emmanuel Benazera
@beniz
Nov 02 2016 21:12
at least theoretically :) getting it to work will bring much joy 😎
Kumar Shubham
@kyrs
Nov 02 2016 21:13
yeah I already have made necessary changes in prototxt . But just for testing purpose I want to train a model with only embed layer and character based encoding without any LSTM layer before going into debugging mode. :)
Emmanuel Benazera
@beniz
Nov 02 2016 21:15
ah. Then the example in beniz/deepdetect#174 might be easier
though note that I've never obtained great scores with the embedding, as far as I remember... The example on news20 should be easy to get started with.
Kumar Shubham
@kyrs
Nov 02 2016 21:18
hmm... ok I will look into it ..thing could have been much easier if we can replicate the IMDB dataset in Keras ...
Emmanuel Benazera
@beniz
Nov 02 2016 21:19
it's the same thing though: remove 18 classes and keep 2
Kumar Shubham
@kyrs
Nov 02 2016 21:20
with character based encoding work in this also ??
*will
Emmanuel Benazera
@beniz
Nov 02 2016 21:21
why do you think it wouldn't or couldn't ?
Kumar Shubham
@kyrs
Nov 02 2016 21:23
:smile: didn't have much success with agnews especially with embed layer that's why ...
Emmanuel Benazera
@beniz
Nov 02 2016 21:24
are you doing a word embedding or a character embedding ?
Kumar Shubham
@kyrs
Nov 02 2016 21:24
character embedding
in my code the lstm is gonna work on top of character embedding ... that's why I was trying to do so
Emmanuel Benazera
@beniz
Nov 02 2016 21:26
yeah embeddings are usually less efficient than bow and boc on small datasets. But they never really did it for me.
the lstm will help define e the embedding via backdrop
first you should get the lstm to forward and backward properly
then second, get the net architecture more optinal
it's difficult working with DD + Caffe, everything after that will look easy.
but the good thing is that you'll know how that works because you had to look into it :)
Kumar Shubham
@kyrs
Nov 02 2016 21:32
sure !! will do that ... this one really have been a a big brainstorming project but yeah learnt a lot of new concepts.
Emmanuel Benazera
@beniz
Nov 02 2016 21:34
ah yes difficult one. In the meantime, the TF branch is now finished for images, so see, takes time but the goal is to never touch the code again after that :)
Will be merged very soon.
Kumar Shubham
@kyrs
Nov 02 2016 21:37
will keep this in mind.. :smile: