These are chat archives for beniz/deepdetect

14th
Nov 2017
rperdon
@rperdon
Nov 14 2017 13:58
My GPU memory is 16GB, when running nvidia-smi doesn't report use anything more than 4gb.
rperdon
@rperdon
Nov 14 2017 14:26
I lowered the batch size for alexnet; seems to be working
its only using 1.5gb of mem now
rperdon
@rperdon
Nov 14 2017 14:43
If I make changes to the prototxt file to add some modifications to the layers, would that be reflected on the re-run of my alexnet model?
Emmanuel Benazera
@beniz
Nov 14 2017 14:47
you'd need to retrain probably.
rperdon
@rperdon
Nov 14 2017 15:34
I altered the default batch size in the prototext file from 256 to 128 and it worked to get it up in running. Initially I had tried a batch size change in the curl call from 32 to 16 but after checking my original prototxt, I was able to modify it to match
rperdon
@rperdon
Nov 14 2017 18:29
WhenI re-run it, does a new deploy.prototxt get updated with my changes to the alexnet.prototxt?
Emmanuel Benazera
@beniz
Nov 14 2017 18:32
No
But I'm not sure what you are doing, you shouldn't need to modify prototxt.
rperdon
@rperdon
Nov 14 2017 18:33
I had a custom alexnet model that altered the bounding areas found for detection
I wanted to test out the differences in the later layers on dd
Emmanuel Benazera
@beniz
Nov 14 2017 18:34
Bounding boxes with alexnet ? Where did you get that, digit ?
rperdon
@rperdon
Nov 14 2017 18:35
I was working with an nvidia researcher to modify the alexnet protofile layers
custom model
yes it was in digits
the original base was alexnet but I suppose I could modify other models with the same changes in layers as well
Emmanuel Benazera
@beniz
Nov 14 2017 18:37
You are riding offroad for DD I think. Use whatever Nvidia uses.
rperdon
@rperdon
Nov 14 2017 18:38
I did get it training; I want to build something that works accurately. googlenet within digits is only giving me a 49% accuracy
googlenet within DD I mean
not digits
Emmanuel Benazera
@beniz
Nov 14 2017 18:38
I'm pretty certain you haven't trained a model with bounding boxes with DD since we train them outside ourselves :)
That's a classification model you might be talking about
rperdon
@rperdon
Nov 14 2017 18:39
yes classification model
Emmanuel Benazera
@beniz
Nov 14 2017 18:39
Ok so no bounding boxes
Or maybe I missed you
rperdon
@rperdon
Nov 14 2017 18:40
the example given within digits seems to outline some bounding boxes
the demo example given was "whale faces"
Emmanuel Benazera
@beniz
Nov 14 2017 18:40
Missread
rperdon
@rperdon
Nov 14 2017 18:40
it was a neat demo of modifying alexnet to find whalefaces in ocean pictures
when I applied the same modifications in digits to alexnet and worked out our dataset, it gave back some really good accuracy
rperdon
@rperdon
Nov 14 2017 19:10
In playing with the learning rates in digits; I was able to get the models to improve their accuracy amounts. With DD I feel like whatever I change doesn't do anything for getting better than a 50/50 split.
Emmanuel Benazera
@beniz
Nov 14 2017 19:17
Training can go wrong for too many reasons. Learning curve is sometimes steep, but worth the trip!
rperdon
@rperdon
Nov 14 2017 19:18
I'll keep working on it. Trying to find that magic number that will coax the model to learn key details is tricky
Emmanuel Benazera
@beniz
Nov 14 2017 19:20
If you have a 2 class problem, it's probably elsewhere. 0.002 is a safe bet almost there everywhere
0.001
rperdon
@rperdon
Nov 14 2017 19:20
I'll try that out if I can get it to stop lol
ctrl-c isn't kicking it out
I ran async false
Emmanuel Benazera
@beniz
Nov 14 2017 19:21
Just don't modify the prototxt by hand. Use resnet_50. Good luck!
rperdon
@rperdon
Nov 14 2017 19:21
I'll try that now that I know the batch size is causing mem problems