by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 28 13:17
    beniz commented #736
  • May 26 12:42
    quadeare synchronize #736
  • May 26 12:38
    quadeare synchronize #736
  • May 26 12:36
    quadeare commented #736
  • May 26 11:37
    beniz commented #736
  • May 25 06:53
    quadeare synchronize #736
  • May 22 07:12
    beniz labeled #734
  • May 22 07:12
    beniz labeled #734
  • May 20 12:28
    quadeare opened #736
  • May 20 09:11
    fantes synchronize #735
  • May 20 09:10
    fantes opened #735
  • May 20 06:55
    fantes synchronize #734
  • May 19 15:10
    fantes edited #734
  • May 19 15:09
    fantes opened #734
  • May 18 09:52
    beniz closed #731
  • May 18 09:52
    beniz commented #731
  • May 18 09:50

    beniz on master

    refactor: rollback Dockerfiles … fix: install cmake version 3.10 chore: rollback build.sh and 2 more (compare)

  • May 18 09:50
    beniz closed #733
  • May 18 09:48
    beniz labeled #733
  • May 18 09:48
    beniz labeled #733
Emmanuel Benazera
@beniz
if you run into trouble, try the code in the demo/ folder as it should work easily
you don't have that many images that you want to use incremental indexing, just index all images, then use a single /predict call with build_index:True to finish the index.
dgtlmoon
@dgtlmoon

@beniz I managed to segfault the server when creating the index, this is not the first time I've seen this happen...

#!/bin/bash

time curl -X POST "http://simsearch_dd:8080/predict" -d '{
       "service":"simsearch",
       "parameters":{
        "input":{ "height": 224, "width": 224  },
         "output":{
  "build_index": true,
  "index_type":"IVF20,SQ8",
  "train_samples":500,
  "ondisk":true,
  "nprobe":10
 },
         "mllib":{ "extract_layer":"fc8"  }
       },
       "data":["/simsearch/5/trimmed/e82db7e9c57185f5e25f2f8b854a42a1.jpg"]
     }'
Segmentation fault (core dumped)

I do not see the build options in the startup only the commit hash...

DeepDetect [ commit b70768a9f327afa024c2747adbf63bfc0e357567 ]
[2020-05-19 16:08:42.464] [api] [info] Running DeepDetect HTTP server on 0.0.0.0:8080

something is not right I think...

i tried searching before I created that index and I did not get any nn results
dgtlmoon
@dgtlmoon
[2020-05-21 10:14:43.462] [api] [info] 172.26.0.3 "POST /predict" simsearch 200 1529
[2020-05-21 10:14:45.508] [api] [info] 172.26.0.3 "POST /predict" simsearch 200 2028
Segmentation fault (core dumped)
dgtlmoon
@dgtlmoon
woha, fc6 seems to work :) :) tho not perfect and in some cases it's easily tricked, but I can see that it does work, amazing :) :), ok trying fc8 again, then will try my own networks and other advanced topics
when i make the DD server segfault, seems its best to restart it to avoid future segfaults
[2020-05-21 11:08:25.797] [api] [info] 172.26.0.3 "PUT /services/simsearch" 201 4890 [2020-05-21 11:08:34.430] [caffe] [info] Opened lmdb /opt/models/vgg16/names.bin open existing index db Segmentation fault (core dumped)
I should create a github issue when I can recreate these segfaults right?
dgtlmoon
@dgtlmoon
vgg16 fc8 works... i dont know what i was doing wrong, i apologise
includes some false positives tho
dgtlmoon
@dgtlmoon
Yup so this same creation/training call works in VGG16 on CPU, but not GPU hmmmm
other errors when trying to pass a whole directory to data[] [2020-05-21 15:42:55.098] [simsearch] [error] Error while proceeding with unsupervised prediction forward pass, not enough memory? src/caffe/blob.cpp:36 / Check failed (custom): (shape[i]) <= (0x7fffffff / count_) [2020-05-21 15:42:55.248] [simsearch] [error] other error: src/caffe/blob.cpp:36 / Check failed (custom): (shape[i]) <= (0x7fffffff / count_)
dgtlmoon
@dgtlmoon
machine has 32gb... [2020-05-21 15:53:56.199] [simsearch] [error] Error while proceeding with unsupervised prediction forward pass, not enough memory? src/caffe/blob.cpp:36 / Check failed (custom): (shape[i]) <= (0x7fffffff / count_) [2020-05-21 15:53:56.304] [simsearch] [error] other error: src/caffe/blob.cpp:36 / Check failed (custom): (shape[i]) <= (0x7fffffff / count_) when I give it a path to my 3200~ images instead of one image at a time
Emmanuel Benazera
@beniz
Hi, make sure you are sending a number of images that can fit in memory.
As for the segfault, sure you can open an issue. List your directory as well. The simsearch is hard tested and in production, but it can be a wrong argument or something with the new faiss db maybe. In all cases we like removing as many seg paths as possible.
Regarding the version, you are correct, it's the commit hash, it's more precise.
dgtlmoon
@dgtlmoon
Heya - I had reasonable decent progress with VGG16 in my simsearch, fc8 looks OK with a little fine tuning, I had trained SqueezeNet to very succesfully find the objects but I could not get any decent output for simsearch with SqueezeNet - Question is: Which layer name from my trained squeezenet would you recommend? it would be better to use my SqueezeNet weights for simsearch as well as object detection right?
dgtlmoon
@dgtlmoon
VGG16 in my simsearch (doing a simsearch of the objects cropped by bbox to image on the disk from the SqueezeNet trained net) was pretty good... maybe 2 out of the top 10 "least distant" results were false-positives, so I guess this is just a fine-tuning issue
I'll setup a test script and see if i can get that to github for you on the weekend
Emmanuel Benazera
@beniz
Hi, you can use the /chain API to get the detection, cropping and simsearch in a single call if you are not already doing this.
dgtlmoon
@dgtlmoon
yeah although, the vgg 16 simsearch output is not good enough for production.. it needs improving/tuning
dgtlmoon
@dgtlmoon
training a ssd300 for object detection and simsearch - hopefully it will give slightly better simsearch results
dgtlmoon
@dgtlmoon

fine tuning a VGG16 to see if it will be a little better at the simsearch ,fc8 gave kind of OK results but needs tuning

[2020-05-23 15:30:29.675] [caffe] [info] Attempting to upgrade input file specified using deprecated V1LayerParameter: /tags_dataset/models/vgg16/VGG_ILSVRC_16_layers.caffemodel
[2020-05-23 15:30:30.530] [caffe] [info] Successfully upgraded file specified using deprecated V1LayerParameter
[2020-05-23 15:30:30.648] [caffe] [info] Ignoring source layer fc8

fc8 is not in model.json... the layer that seemed to work good for simsearch

and my vgg16 recipe https://gist.github.com/dgtlmoon/b6f772c181d8d232b4910f4c19765750 gives [2020-05-23 15:35:07.316] [api] [error] {"code":500,"msg":"InternalError","dd_code":500,"dd_msg":"solver creation exception"} ... dont know how to finetune then
Emmanuel Benazera
@beniz
fc8_ftune probably because of finetuning
dgtlmoon
@dgtlmoon
@beniz ahuh! thanks again... sorry for the mindless questions... :) that solver creation exception i dont know where to even start, not much on google about it
recipe above in the gist
dgtlmoon
@dgtlmoon
Finetuned ssd300 as an object detector (works OK tho squeezenet was better), loaded 3200 similar images, extract_layer: fc6 I have 2.8Gb (!) of index_mmap.faiss index, prediction simsearch call gives me 0 similar images :/ I need to figure out that solver creation exception in vgg16 i think... train samples was "train_samples": 100,... or use squeezenet as simsearch also, but i cant find a good layername to use
dgtlmoon
@dgtlmoon
1.3Mb of faiss index per image :D
dgtlmoon
@dgtlmoon
trying indexing pool10 from my fairly succesfull squeezenet finetuned objectdetector as simsearch now
YaYaB
@YaYaB

Hey guys,
I am trying to build DD on the latest commit (fc01676065cec815deecb629256fa968b0442e0b) with tensorrt-oss and tensort5 however I quickly run in the following error.

Scanning dependencies of target tensorrt-oss
[  2%] Creating directories for 'tensorrt-oss'
[  4%] Performing download step (git clone) for 'tensorrt-oss'
Cloning into 'origin'...
CMake Error at /home/NONAME/deepdetect/build_last/tensorrt-oss/tmp/tensorrt-oss-gitclone.cmake:49 (message):
  Failed to checkout tag: '0d36bbb29732cdefbed6a60b51039ea1fa747742'


CMakeFiles/tensorrt-oss.dir/build.make:88: recipe for target 'tensorrt-oss/src/tensorrt-oss-stamp/tensorrt-oss-download' failed
make[2]: *** [tensorrt-oss/src/tensorrt-oss-stamp/tensorrt-oss-download] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/tensorrt-oss.dir/all' failed
make[1]: *** [CMakeFiles/tensorrt-oss.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

I modified a bit the /home/NONAME/deepdetect/build_last/tensorrt-oss/tmp/tensorrt-oss-gitclone.cmake by changing "/origin" to "origin". It is able to checkout then however it fails with

error: pathspec 'origin.detachedHead=false' did not match any file(s) known to git.
CMake Error at /home/NONAME/deepdetect/build_last/tensorrt-oss/tmp/tensorrt-oss-gitclone.cmake:58 (message):
  Failed to init submodules in: '/origin'

Could you help me with that?

dgtlmoon
@dgtlmoon
@YaYaB can you git clone https://github.com/NVIDIA/TensorRT as a test from commandline? does it work for you?
Emmanuel Benazera
@beniz
@YaYaB maybe their branch names has changed, you can look it up and report it in an issue if that is the case
YaYaB
@YaYaB

@YaYaB can you git clone https://github.com/NVIDIA/TensorRT as a test from commandline? does it work for you?

Yep it does work

@YaYaB maybe their branch names has changed, you can look it up and report it in an issue if that is the case

In local I can access the correct release

TensorRT git/master  
❯ git checkout 0d36bbb29732cdefbed6a60b51039ea1fa747742                                                                                                                                                         
Note: checking out '0d36bbb29732cdefbed6a60b51039ea1fa747742'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 0d36bbb... TensorRT Open Source Release/5.1
Emmanuel Benazera
@beniz
@YaYaB the DD build with TRT OSS works for me, cmake .. -DUSE_CUDNN=ON -DUSE_TENSORRT=ON -DUSE_TENSORRT_OSS=ON
@dgtlmoon post your full call if you need help with training and simsearch
YaYaB
@YaYaB
@beniz Those are the options I use to build, are you on the latest commit?
Emmanuel Benazera
@beniz
ah maybe not, but I can retry and I'll let you know
YaYaB
@YaYaB
That would be nice, thanks!
Emmanuel Benazera
@beniz
build appears to work for me
YaYaB
@YaYaB
hum very akward :O. What is your version of cmake? I am on 3.5.1
Emmanuel Benazera
@beniz
3.14.0
YaYaB
@YaYaB
Yep I just tried with 3.14.0 and it seems to compile correctly
Emmanuel Benazera
@beniz
CUDA requires cmake 3.14 minimum I believe, it's on the build page instructions for 18.04.
YaYaB
@YaYaB
Yep but I am on 16.04 ^^
Emmanuel Benazera
@beniz
same for 16.04
otherwise their cmake detection script for CUDA doesn't work
YaYaB
@YaYaB
Hum without tensort-oss it works without any flaw with a lower version of cmake