thanks for the information, the coordinates should be fine i tested some of them. But how do u test them on the platform? I couldn't find this ui from the guide
Hello @tasibalint , this requires installing the platform. If your bbox are fine, stay with your scripts, and try using the
weights from this model instead of your current .caffemodel : https://www.deepdetect.com/models/detection_600/
also, put the
prob of the distort object to 0.5
and put
rgb to false
Can train now with detection_600 amma post my calls so people might find it in the future:
Create Service JSON:
{
"description":"generic image detection service",
"model":{
"repository":"/images/models/packages_detc",
"create_repository": true,
"templates":"../templates/caffe/",
"weight": "VGG_openimage_pretrain_ilsvrc_res_pred_openimage_detect_v2_SSD_openimage_pretrain_ilsvrc_res_pred_300x300_iter_1200000.caffemodel"
},
"mllib":"caffe",
"type":"supervised",
"parameters":{
"input":{
"connector":"image",
"width":300,
"height":300,
"db": true,
"bbox": true
},
"mllib":{
"template":"vgg_16",
"nclasses":602,
"finetuning":true
}
},
}Start training:
{
"service":"packages_detc",
"async":true,
"parameters":{
"input":{
"connector":"image",
"shuffle":true,
"height":300,
"width":300,
"db":true,
"rgb":false,
},
"mllib":{
"gpu":false,
"mirror":true,
"net":{
"batch_size":5,
"test_batch_size":5
},
"solver":{
"test_interval":250,
"iterations":1000,
"solver_type": "ADAM"
},
"noise":{"all_effects":true, "prob":0.001},
"distort":{"all_effects":true, "prob":0.5},
"bbox": true
},
"output":{
"measure":["map"],
}
},
"data":["/images/train/train.txt", "/images/train/train.txt"]
}
Ouch something is still not working :/ training started running but failed here
I dunno anymore I am going to wait until i get a response from you guys, what template i could use in order to train the detection 600 weight
@tasibalint yes this works too, use
refinedet_512 and download https://www.deepdetect.com/models/pretrained/refinedet_512/VOC0712_refinedet_vgg16_512x512_iter_120000.caffemodel to use them as weights
you may need to lower
batch_size and iter_size depending on your GPU. You can set iter_size to 4 to compensate. You'll need many more than 1000 iterations also, not sure what your data are, but 25000 is reasonable for a first run/try.
your bbox's are in the wrong format, follow the tutorial again, i think you have a mistake in the layout of your text files
My image_32.txt files are build like:
<label> <xmin> <ymin> <xmax> <ymax>
601 22 32 673 756
601 because detection 600 had 600 classes in it and i wanted a new class
train.txt lines are :
[docker volume folder]/train/image_3.jpg- [docker volume folder]/train/image_3.txt
What do you mean wrong format?
@beniz getting this error since i used the last model u posted
Any idea as to what I should do?
On the other hand, when I use an online image (its url) the api returns the predictions I wanted
hey @beniz - happy new year! long time since I've been looking through this stuff - tons of updates!! was just wondering if you're still building docker images with TF included. I'm trying to update our build process and I'm running into quite a number of issues with changes in DD affecting the build as well as changes in floopz/tensorflow_cc (and tensorflow itself).
Anyways, was just wondering if a (even CPU-only, for now) automated build is still happening with tensorflow that I can compare my build process too. Thanks in advance!
I see, that's kind of what I figured :+1: are you still using caffe for anything? I know it's the default backend for builds, but curious if you're actually using it.
Also, do you have a reliable pipeline for converting models? I'd love to convert some of our models but there seems to be quite a bit of manual effort involved (not to mention validating/testing afterwards). If you have any sort of pipeline I'd be very interested :)
Also, do you have a reliable pipeline for converting models? I'd love to convert some of our models but there seems to be quite a bit of manual effort involved (not to mention validating/testing afterwards). If you have any sort of pipeline I'd be very interested :)
@cchadowitz-pf we don't train new models with caffe, the 'legacy' models still running with our customers are in practice running the tensorrt backend (that got a nice upgrade recently in DD), thus abstracting away the initial backend. In your case it's a bit different since you are running CPU, correct ?
I believe the best pipeline for CPU would be to convert to ONNX and use onnxruntime. You could use onnxruntime directly, or we could add it to DD. It's actually on our optional todolist, but as we are driven by our customers and they never asked for CPU we didn't got it into DD. Not difficult for inference though.
Let me know your thoughts.
I believe the best pipeline for CPU would be to convert to ONNX and use onnxruntime. You could use onnxruntime directly, or we could add it to DD. It's actually on our optional todolist, but as we are driven by our customers and they never asked for CPU we didn't got it into DD. Not difficult for inference though.
Let me know your thoughts.
I see :+1: we definitely still have CPU in some cases unfortunately. Any reason you suggest using onnxruntime instead of libtorch? Is it just more efficient without a GPU?
In practice, it sounds like you're more often training models and then deploying, rather than converting models - is that accurate? It sounds like I'll need to put together a methodology to compare and validate models before/after converting them. I'm hoping to not have to do it manually but it will probably come down to that....
In practice, it sounds like you're more often training models and then deploying, rather than converting models - is that accurate? It sounds like I'll need to put together a methodology to compare and validate models before/after converting them. I'm hoping to not have to do it manually but it will probably come down to that....
and yes! many new and awesome things lately in DD - that's part of why I'm trying to get our build up to date again hah. Have yet to find a good way to continue to use the TF openimages pretrained model outside of TF, however.
Hi @cchadowitz-pf , suggesting onnxruntime since you are asking about converting your tf models: converting to onnx seems more appropriate than TF to pytorch, but you may want to double check. For simple models such as openimage, this seems a no brainer, but in reality... who know ?!
onnxruntime reports good performances also.
You are correct we are always training custom models, that's actually our business. If the openimage model is your sole TF dependency, you may be able to convert it or find a more recent and better one for torch.
I'll keep looking around but may pop back in here for more thoughts :)