These are chat archives for beniz/deepdetect

20th
Apr 2017
Bas van Boven
@BasVanBoven
Apr 20 2017 08:17
Hey @beniz , I was kindly wondering if you found anything out regarding issue #290 since it was now labeled as a 'Bug' :). Does that mean the GPU Docker container is the cause of the issue, or should I investigate further?
Emmanuel Benazera
@beniz
Apr 20 2017 08:20
@BasVanBoven we're on it, stay tune ;)
Bas van Boven
@BasVanBoven
Apr 20 2017 08:21
Thank you very much! I will not jump the DeepDetect ship then for my project ;)
If there is anything I can do to help, let me know :)
Emmanuel Benazera
@beniz
Apr 20 2017 08:43
it should be ready sometimes this morning, cross fingers...
Bas van Boven
@BasVanBoven
Apr 20 2017 08:45
I will!
Emmanuel Benazera
@beniz
Apr 20 2017 09:53
FYI, we've got it running, seems like new nvidia-docker + cudnn did not work well for us, so we've put some more containers to build to see if the auto-build returns a fixed container, and once we get a positive answer to that, we'll push a fresh one online
Bas van Boven
@BasVanBoven
Apr 20 2017 09:56
So happy that you got it working! Saves me quite some headache to have identified the problem, haha. I will be looking out for the container and test it when it is possible to do so.
Emmanuel Benazera
@beniz
Apr 20 2017 11:14
@BasVanBoven OK, so look for beniz/deepdetect_gpu_pascal, it's being uploaded at the moment, so you may have to wait a few minutes. Let us know how this goes, it appears to work fine on our side on Pascal arch.
Bas van Boven
@BasVanBoven
Apr 20 2017 11:19
Thanks a lot! I will see if I can reach our deployment server in Singapore (its EOD for them) and report back ASAP.
Bas van Boven
@BasVanBoven
Apr 20 2017 12:25
Good news: I have pulled in the new Docker container and it works flawlessly on the GTX 1080! :)
Many thanks for your support, it has been a great help!
Emmanuel Benazera
@beniz
Apr 20 2017 13:42
peace ;)
cchadowitz-pf
@cchadowitz-pf
Apr 20 2017 13:49
Hi @beniz, just poking you to see if you have any further thoughts about #289 :) for now I've simply reverted to disabling all gpu use, but I wasn't sure if poking around for #290 gave you any further ideas ;)
Emmanuel Benazera
@beniz
Apr 20 2017 14:12
We haven't looked at it yet. We (and I mostly when I can) do fast track the bugs (and having the GPU docker was one), but we can be slower at features.
cchadowitz-pf
@cchadowitz-pf
Apr 20 2017 14:14
No problem, I understand :) I'm guessing most DeepDetect users either don't use GPU, or don't mix caffe and TF models, or have separate instances, so aren't as affected by this. Thanks again for the help!
Emmanuel Benazera
@beniz
Apr 20 2017 14:15
We will look at it, just a matter of time. Separating the builds is useful
cchadowitz-pf
@cchadowitz-pf
Apr 20 2017 14:19
yeah - do you have any need/desire for any of the dockerfiles I have in use at the moment (related and unrelated to #289)? I noticed that the official ones don't include TF. I've streamlined the process for Ubuntu 16.04 on CPU (with TF), and I also have a mostly streamlined process for GPU, with the caveats regarding the shared GPU issue I'm dealing with :)
Emmanuel Benazera
@beniz
Apr 20 2017 14:21
yes, contributing docker with TF build would be useful
we actually still have issues with the docker GPU automatic builds, the one I've pushed earlier was partly fixed by hand.
cchadowitz-pf
@cchadowitz-pf
Apr 20 2017 14:23
I have docker GPU builds working from dockerfiles (with and without TF), though I'm not 100% that my approach is necessarily the 'correct' approach
I can open a pull request for you to take a peek at
Emmanuel Benazera
@beniz
Apr 20 2017 14:34
sure, that'd be great! what GPU model are you using ? (the build may differ)
cchadowitz-pf
@cchadowitz-pf
Apr 20 2017 14:37
currently primarily GeForce GTX 1070 (driver 375.20), which I believe is CUDA arch 6.1, but also for GeForce GTX TITAN X (driver 375.26), which I believe is CUDA arch 5.2. Some of my changes may be specific to my setup, but it may give you some hints if nothing else
(all on Ubuntu 16.04, CUDA 8.0, cuDNN 5.1)
Emmanuel Benazera
@beniz
Apr 20 2017 14:42
that's OK. btw, cudnn 6 is out. Not sure whether it changes much though :)
cchadowitz-pf
@cchadowitz-pf
Apr 20 2017 14:43
i did see that, but i also saw one or two references of it being slower than 5.1 ;) i don't think it will change much though tbh
Emmanuel Benazera
@beniz
Apr 20 2017 15:45
@BasVanBoven hey, if you could test with the new beniz/deepdetect_gpu, that'd be great. Thanks.
no hurry though.
cchadowitz-pf
@cchadowitz-pf
Apr 20 2017 18:37
@beniz check out #292 for my Dockerfiles. I tried to stay as true to the official ones as I could, but let me know if you have any questions or concerns