These are chat archives for beniz/deepdetect

10th
Jul 2018
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 14:47
hi @beniz! is it possible that a change in the last week or so has changed things for Caffe services where if a request specifies "bbox": true for a model that may or may not in fact provide bounding boxes, it no longer returns 0 for bbox coordinates and instead returns values like 1.13608909679294e30, 1.540683774395253e23, 4.2778839518908018e-41,3.5953860554229104e308?
Emmanuel Benazera
@beniz
Jul 10 2018 14:51
it shouldn't
Emmanuel Benazera
@beniz
Jul 10 2018 14:56
just checked on one of our up to date instances and it seems OK
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 14:57
strange. a caffe model that as far as i know doesn't use bounding boxes used to return 0 for all coordinates even when bbox: true was set, now it's returning values as above....
Emmanuel Benazera
@beniz
Jul 10 2018 14:58
have you updated a docker or something ? we're using the last docker
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:00
yeah i have.....just yesterday actually
Emmanuel Benazera
@beniz
Jul 10 2018 15:00
there are new ones from this morning, so that at least it's comparable to ours.
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:01
new jolibrain images or docker engine runtime?
Emmanuel Benazera
@beniz
Jul 10 2018 15:01
image
if you still see this, post your creation and prediction calls
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:01
hmm ok. i'm technically building customized ones locally from the github master branch but i'll make sure to check out the latest commit and try again to be sure
yup will do
thanks!
Emmanuel Benazera
@beniz
Jul 10 2018 15:02
ok
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:03
fwiw the model in question is "VGG16-hybrid1365" from https://github.com/CSAILVision/places365
Emmanuel Benazera
@beniz
Jul 10 2018 15:04
is this an object detection model ?
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:04
i didn't think so :smile:
Emmanuel Benazera
@beniz
Jul 10 2018 15:05
why the bbox flag ?
you shouldn't use it I think
otherwise the output is wrong
as it's looking for stuff within the blobs that it can't find
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:06
i'm automatically hitting a number of deepdetect services and some are detection models some are not, so at the time it was simpler to simply put the bbox flag in for all. originally it did not cause any issues as models that were not detection models returned bboxes of 0,0,0,0
was curious why that changed now :)
Emmanuel Benazera
@beniz
Jul 10 2018 15:06
the status call outputs an mltype variable that holds the model type
info call
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:06
yup i saw that was added in
Emmanuel Benazera
@beniz
Jul 10 2018 15:06
try that
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:07
{
  "status": {
    "code": 200,
    "msg": "OK"
  },
  "body": {
    "mltype": "classification",
    "name": "placeshybrid",
    "parameters": {
      "mllib": [
        {
          "gpu": true,
          "nclasses": 1365
        }
      ],
      "input": [
        {
          "height": 224,
          "width": 224,
          "connector": "image"
        }
      ]
    },
    "description": "placeshybrid",
    "mllib": "caffe",
    "jobs": [],
    "predict": true
  }
}
the correct thing to do would be for me to only set the bbox flag for models i know provide bboxes (detection models), absolutely. i was just curious why non-detection models were returning erroneous values now
to be clear, if i set the bbox flag during service creation, i shouldn't need to set it for subsequent predict calls, right?
Emmanuel Benazera
@beniz
Jul 10 2018 15:11
not really, as it's an output parameter, it's not carried across from service creation
so at the moment it's volatile
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:12
oh, so i should still set it for predict calls? i was looking at the api (https://deepdetect.com/api/#create-a-service) and it seems to indicate that setting it at service creation matters bbox bool yes false whether to setup an object detection model
there it seems to be listed under the image input connector
Emmanuel Benazera
@beniz
Jul 10 2018 15:12
it's for training, it's been added recently, as part of the input parameter
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:12
ahhh. gotcha.
Emmanuel Benazera
@beniz
Jul 10 2018 15:13
yeah, not forcely very clear. the website will be rebuilt with thorough novel documentation. We're just too busy elsewhere on delivery at the moment
so code is ahead of the documentation, though the API is kept up to date
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:14
:+1: no worries. i'll make sure to only use the bbox flag when i know it's a detection model in the future, was just hoping there was a simple reason why the outputs changed regardless :)
Emmanuel Benazera
@beniz
Jul 10 2018 15:15
not sure why it did change for you, I'm actually not able to predict what would be the outcome of looking for bbox from a non detection model!
(we could prevent a crash if this was likely)
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 15:16
:D i was under the impression previously that the bbox output data object would just be filled with the default 0's, but maybe now somehow values are coming out of caffe so they're no longer default 0
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 16:10
some more info about this weird situation - if i do a predict call with a single image, it returns 0,0,0,0 for the bounding box coords (for models that don't do detection, with bbox flag enabled). if i do a batch of a number of images in the same scenario as above, some come back with 0,0,0,0, some come back with these strange values. the odd bit is that the single image that returned 0,0,0,0 on its own came back with different values for the bbox coords when included in a larger batch
so maybe something is getting shuffled around when processing a batch and producing these erroneous values?
service creation call:
curl -X PUT "http://localhost:8080/services/placeshybrid" -d '{
  "mllib":"caffe",
  "description":"placeshybrid",
  "type":"supervised",
  "parameters":{
    "input":{
      "connector":"image",
      "width":224,
      "height":224
    },
    "mllib":{
      "nclasses":1365,
      "gpu":true
    }
  },
  "model":{
    "repository":"/opt/models/placeshybrid/"
  }
}'
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 16:16

single image predict call:

curl -X POST "http://localhost:8080/predict" -d '{
"service": "placeshybrid", "parameters":{ "output": {"bbox": true, "best": 1}}, "data": ["http://localhost:8000/sample_001726-resize1024.jpg"]}'

batched images predict call:

curl -X POST "http://localhost:8080/predict" -d '{
"service": "placeshybrid", "parameters":{ "output": {"bbox": true, "best": 1}}, "data": ["http://localhost:8000/sample_000106-resize1024.jpg","http://localhost:8000/sample_000196-resize1024.jpg","http://localhost:8000/sample_000286-resize1024.jpg","http://localhost:8000/sample_000376-resize1024.jpg","http://localhost:8000/sample_000466-resize1024.jpg","http://localhost:8000/sample_000556-resize1024.jpg","http://localhost:8000/sample_000646-resize1024.jpg","http://localhost:8000/sample_000736-resize1024.jpg","http://localhost:8000/sample_000826-resize1024.jpg","http://localhost:8000/sample_000916-resize1024.jpg","http://localhost:8000/sample_001006-resize1024.jpg","http://localhost:8000/sample_001096-resize1024.jpg","http://localhost:8000/sample_001186-resize1024.jpg","http://localhost:8000/sample_001276-resize1024.jpg","http://localhost:8000/sample_001366-resize1024.jpg","http://localhost:8000/sample_001456-resize1024.jpg","http://localhost:8000/sample_001546-resize1024.jpg","http://localhost:8000/sample_001636-resize1024.jpg","http://localhost:8000/sample_001726-resize1024.jpg","http://localhost:8000/sample_001816-resize1024.jpg"]}'

the single image predict call returns:

"bbox": {
              "xmax": 0,
              "ymax": 0,
              "ymin": 0,
              "xmin": 0
            }

and the batched images predict call for that same file (sample_001726-resize1024.jpg) returns:

"bbox": {
              "xmax": 1.7976931348623157e+308,
              "ymax": 2.281771322195977e+31,
              "ymin": 3.874309994165254e-41,
              "xmin": 1.4349296274686127e-42
            }
Emmanuel Benazera
@beniz
Jul 10 2018 16:23
It's looking through output blobs that are of the wrong dimensions. Behavior is unexpected at best
cchadowitz-pf
@cchadowitz-pf
Jul 10 2018 16:23
:+1:
another indicator of what you're saying - the probability and label are totally wrong as well if bbox flag is set incorrectly

same file, same model:
with bbox: true:

{
            "prob": 0,
            "last": true,
            "bbox": {
              "xmax": 0,
              "ymax": 0,
              "ymin": 0,
              "xmin": 0
            },
            "cat": "tench, Tinca tinca"
          }

with bbox: false:

{
            "prob": 0.7298522591590881,
            "last": true,
            "cat": "amphibian, amphibious vehicle"
          }
thanks!