Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 29 18:31
    mergify[bot] unlabeled #1446
  • Jun 29 18:31
    mergify[bot] closed #1446
  • Jun 29 18:31

    mergify[bot] on master

    chore: update to oatpp 1.3 (compare)

  • Jun 29 18:31
    mergify[bot] labeled #1446
  • Jun 29 15:45
    Bycob synchronize #1446
  • Jun 29 10:35
    Bycob synchronize #1446
  • Jun 28 15:54
    Bycob synchronize #1446
  • Jun 27 11:49
    Bycob labeled #1446
  • Jun 27 08:57
    Bycob synchronize #1446
  • Jun 27 07:54
    beniz commented #1446
  • Jun 25 09:18
    chichaj unassigned #547
  • Jun 25 09:18
    chichaj unassigned #443
  • Jun 25 09:18
    chichaj unassigned #447
  • Jun 24 16:00
    mergify[bot] review_requested #1446
  • Jun 24 16:00
    mergify[bot] review_requested #1446
  • Jun 24 15:59
    Bycob labeled #1446
  • Jun 24 15:59
    Bycob labeled #1446
  • Jun 24 15:59
    Bycob opened #1446
  • Jun 22 15:41
    Bycob edited #1437
  • Jun 22 10:44
    mergify[bot] unlabeled #1437
dgtlmoon
@dgtlmoon
hmm I need to work on a pipeline that builds one huge image with lots of example simsearch results, so that I can better visualise if i'm improving or not, then try different layers too
Emmanuel Benazera
@beniz
Hi @dgtlmoon by looking at results it's easier to "debug", diagnosis and improve
dgtlmoon
@dgtlmoon
hey :) yeah.. ok, i'll focus on making (automated) some giant JPEG with result comparisons
Emmanuel Benazera
@beniz
Also a clear definition of what similar should mean in your context could be useful since you are looking to optimize further.
dgtlmoon
@dgtlmoon
yup for sure :) i'll get back to you with something more concrete :) some nice big reports or something
dgtlmoon
@dgtlmoon
@beniz training object detector question - when i'm training multiple classes, and those classes/objects generally always appear in the same image, should I always be sure to train an image that contains all of the objects? or is it totally no difference if I have a dataset of Object A (bbox txt's and imgs) and a dataset of Object B (bbox txt's and imgs), but in practice the images i'm detecting on always contain both A and B? should be no difference in accuracy or?
dgtlmoon
@dgtlmoon
congrats on v0.20.0 :D
dgtlmoon
@dgtlmoon
[2022-02-04 16:06:58.042] [torchlib] [info] Initializing net from parameters: 
[2022-02-04 16:06:58.042] [torchlib] [info] Creating layer / name=tdata / type=AnnotatedData
[2022-02-04 16:06:58.042] [torchlib] [info] Creating Layer tdata
[2022-02-04 16:06:58.042] [torchlib] [info] tdata -> data
[2022-02-04 16:06:58.042] [torchlib] [info] tdata -> label
terminate called after throwing an instance of 'CaffeErrorException'
  what():  ./include/caffe/util/db_lmdb.hpp:15 / Check failed (custom): (mdb_status) == (0)
 0# dd::OatppJsonAPI::abort(int) at /opt/deepdetect/src/oatppjsonapi.cc:255
 1# 0x00007F24AC053210 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise in /lib/x86_64-linux-gnu/libc.so.6
 3# abort in /lib/x86_64-linux-gnu/libc.so.6
 4# 0x00007F24AC477911 in /lib/x86_64-linux-gnu/libstdc++.so.6
 5# 0x00007F24AC48338C in /lib/x86_64-linux-gnu/libstdc++.so.6
 6# 0x00007F24AC4833F7 in /lib/x86_64-linux-gnu/libstdc++.so.6
 7# 0x00007F24AC4836A9 in /lib/x86_64-linux-gnu/libstdc++.so.6
 8# caffe::db::MDB_CHECK(int) at ./include/caffe/util/db_lmdb.hpp:15
 9# caffe::db::LMDB::Open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, caffe::db::Mode) at src/caffe/util/db_lmdb.cpp:40
10# caffe::DataReader<caffe::AnnotatedDatum>::Body::InternalThreadEntry() at src/caffe/data_reader.cpp:95
11# 0x00007F24B0DC343B in /lib/x86_64-linux-gnu/libboost_thread.so.1.71.0
12# start_thread in /lib/x86_64-linux-gnu/libpthread.so.0
13# __clone in /lib/x86_64-linux-gnu/libc.so.6

Aborted (core dumped)

current jolibrain/deepdetect_cpu with https://www.deepdetect.com/downloads/platform/pretrained/ssd_300/VGG_rotate_generic_detect_v2_SSD_rotate_300x300_iter_115000.caffemodel

used to work before i think.. checking GPU version

hmm maybe related to a permissions issue in the model dir? src/caffe/util/db_lmdb.cpp LMDB etc related.. maybe.. checking
x/detection$ ls -al model/train.lmdb/
total 29932
drwxr--r-- 2 dgtlmoon dgtlmoon     4096 Feb  4 17:10 .
drwxrwxrwx 3 dgtlmoon dgtlmoon     4096 Feb  4 17:10 ..
-rw-r--r-- 1 dgtlmoon dgtlmoon 30638080 Feb  4 17:10 data.mdb
-rw-r--r-- 1 dgtlmoon dgtlmoon     8192 Feb  4 17:10 lock.mdb
looks atleast like its writing, I'm nuking that dir when i create the service, so these are always fresh
Emmanuel Benazera
@beniz
@dgtlmoon it doesn't matter if some objects are on some images only if this is your question
make sure you have the writing permissions overall since this is being access by the docker image.
dgtlmoon
@dgtlmoon
@beniz i'm doing docker stuff for many years now, that was the first thing i checked :)
dd@5c643a8f4b39:/tags_dataset/bottom/model$ mkdir foobar
dd@5c643a8f4b39:/tags_dataset/bottom/model$ touch foobar/ok.txt
dd@5c643a8f4b39:/tags_dataset/bottom/model$
I can see that train.lmdb/data.mdb is created without problem brand new every time, and THEN the segfault happens
the setup of the service works fine, then I call the train method, I see the MDB is created every time, and i get segfault

curl -X POST "http://localhost:8080/train" -d '
{
  "service": "location",
  "async": true,
  "parameters": {
    "input": {
      "db": true,
      "db_width": 512,
      "db_height": 512,
           "width": 300,
           "height": 300

    },
    "mllib": {
      "resume": false,
      "net": {
        "batch_size": 20,
        "test_batch_size": 12
      },
      "solver": {
        "iterations": 50000,
        "test_interval": 500,
        "snapshot": 1000,
        "base_lr": 0.0001
      },
      "bbox": true
    },
    "output": {
      "measure": [
        "map"
      ]
    }
  },
  "data": [ "/tags_dataset/bottom/bottom-images.txt" ]
}
'
dgtlmoon
@dgtlmoon
I even chmod 777'ed the model dir just to be safe
dgtlmoon
@dgtlmoon
hmm ok, tried some earlier tags like v0.15.0 v0.17.0 which I remember worked, same error, so I'm doing something weird here somehow
dgtlmoon
@dgtlmoon

got it.. needs some check put in the code I think

Forgot to add test AND train lists

  "data": [ "/tags_dataset/bottom/train.txt" ]

will cause the crash

Emmanuel Benazera
@beniz
ah thanks, this should be caught by unit tests, and an error returned instead.
dgtlmoon
@dgtlmoon
done :)
dgtlmoon
@dgtlmoon

I love when you get simple problems to solve that goto amazing accuracy in a few iterations

"map_hist": [ 0.5956036100784937, 0.8226678265879551, 0.9506028971324364, 0.9687120678524176 ] :)

dgtlmoon
@dgtlmoon
I tried using a single file of images to train on, and added "parameters": { "input": { "test_split": 0.10 to both service setup and service train calls, but I still get that same segfault
Emmanuel Benazera
@beniz
object detector training with caffe has no test_split
dgtlmoon
@dgtlmoon
x)
Emmanuel Benazera
@beniz
it's going to be deprecated soon, for the torch backend with object detection. At the moment, split by hand and provide a test.txt file, the fix has been merged.
dgtlmoon
@dgtlmoon
ok I usually use caffe for absolutely no reason than that's whats in the examples, maybe I should try torch? (trying squeezenet object detector here)
yes I have a python script to split stuff for me I wrote
that's funny, so actually the bug I found was completely unrelated to what I was doing :D
dgtlmoon
@dgtlmoon
I wonder if you can run a classifier with just one class, i'll try, i think so
Nur
@nurkbts
Hi
How can use code for AI Server , easy setup
Marc Reichman
@marcreichman
Hello @beniz or anyone. Quick question - for a pretrained scenario with just classifications, is it possible to run in read-only docker? I am seeing a message couldn't write model.json file in model repository... when loading, but classifications seem to work fine. Can this message be disregarded in this case?
Emmanuel Benazera
@beniz
Hey @marcreichman yes I believe you can ignore.
Marc Reichman
@marcreichman
Thanks @beniz !