These are chat archives for beniz/deepdetect

5th
Nov 2016
ProFire
@ProFire
Nov 05 2016 06:19
Hello everyone. I'm quite new to DD. I have a school assignment where I have to predict number sequence. 0.1, 0.2, 0.4, 0.8, what's the next number in sequence? I am not allowed to use human intelligence to answer this question, but must use ML. What would be most suitable tutorial for me to follow?
Emmanuel Benazera
@beniz
Nov 05 2016 06:24
@ProFire I don't think you want DD for this as it is more of a production tool. Look at scikit-learn and keras instead I'd say.
ProFire
@ProFire
Nov 05 2016 06:28
I have tried scikit-learn and it is wonderful. But I have a catch: I needed something that is of API so that my fellow cohort can test against my prediction, likewise, I test others. We have a sort of ML competition. My search on Google led me to DD. I'm hoping to utilise DD for this assignment.
There will be other number sequence, such as 0761 = 2, 9856 = 3, what is 5746 equals to?
Emmanuel Benazera
@beniz
Nov 05 2016 06:30
this should work well enough if your sequences are fixed size, otherwise I'd advise to go keras and use LSTM
ProFire
@ProFire
Nov 05 2016 06:33
Oh thank you. I'll give that a try. I've already installed DD with docker. I hope I get good results. My sequences so far are quite fixed size. If I am given another interesting assignment, I'll come back here for advice. =D
Emmanuel Benazera
@beniz
Nov 05 2016 06:33
good luck ^^
ProFire
@ProFire
Nov 05 2016 06:40
The training stopped when I followed the xgb tutorial. I checked the logs and this error shows: {"code":500,"msg":"InternalError","dd_code":1007,"dd_msg":"[06:38:49] src/io/local_filesys.cc:154: Check failed: allow_null LocalFileSystem: fail to open \"models/covert/0100.model\""}
Did I make a mistake somewhere?
Emmanuel Benazera
@beniz
Nov 05 2016 06:41
yeah, using docker you need to share your training repository with the container, it's in the README I think
the server is within the container and cannot look outside to the host unless you make a bridge between the two
look the 'sharing a volume with the image' section up
ProFire
@ProFire
Nov 05 2016 06:44
I followed https://github.com/beniz/deepdetect/tree/master/docker and used this command to start the container: docker run -d -p 8080:8080 -v /home/profire:/root beniz/deepdetect_cpu
When I followed the tutorial, I was careful to "data":["/root/train.csv"] as I placed it there.
Could I have made a different error from your suggestion?
Emmanuel Benazera
@beniz
Nov 05 2016 06:46
I don't think you want to attach your volume to /root ...
though maybe...
ProFire
@ProFire
Nov 05 2016 06:46
I'm not sure if this helps. This is in the log: include/dmlc/logging.h:235: [06:38:49] src/io/local_filesys.cc:154: Check failed: allow_null LocalFileSystem: fail to open "models/covert/0100.model"
Emmanuel Benazera
@beniz
Nov 05 2016 06:47
sounds like the path to the models is incomplete
what are your API calls
ProFire
@ProFire
Nov 05 2016 06:48

curl -X PUT "http://localhost:8080/services/covert" -d '{
"mllib":"xgboost",
"description":"forest classification service",
"type":"supervised",
"parameters":{
"input":{
"connector":"csv"
},
"mllib":{
"nclasses":7
}
},
"model":{
"repository":"models/covert"
}
}'

and

curl -X POST "http://localhost:8080/train" -d '{
"service":"covert",
"async":true,
"parameters":{
"mllib":{
"iterations":100,
"test_interval":10,
"objective":"multi:softprob"
},
"input":{
"label_offset":-1,
"label":"Cover_Type",
"id":"Id",
"separator":",",
"shuffle":true,
"test_split":0.1
},
"output":{
"measure":["acc","mcll","f1"]
}
},
"data":["/root/train.csv"]
}'

I'm running on CPU with 4GB ram.
Emmanuel Benazera
@beniz
Nov 05 2016 06:49
set a an absolute path as repository
ProFire
@ProFire
Nov 05 2016 06:49
Ok. I'll give that a try.
Emmanuel Benazera
@beniz
Nov 05 2016 06:53
relative paths are always wrt to the server starting point, so with docker it's rather awkward
ProFire
@ProFire
Nov 05 2016 06:55
I still got the same error. This time round, during service creation, I used /opt/models/convert
In the logs, I see that it boosted till 99, when the error occured
Emmanuel Benazera
@beniz
Nov 05 2016 06:56
you should put the model onto your shared volume space and thus you should be able to see the files on the host side
ProFire
@ProFire
Nov 05 2016 06:57
I see... I'll try again.
ProFire
@ProFire
Nov 05 2016 07:02
I get this error this time round: src/io/local_filesys.cc:154: Check failed: allow_null LocalFileSystem: fail to open "/root/models/covert/0100.model"
I've ensured that the repository is /root/models/covert
Could it be that I should not have used /root?
I noticed the folder is empty on the host machine.
ProFire
@ProFire
Nov 05 2016 07:19
I just relaunched docker with /home/profire:/home/dd as the bridge
Then did the API to the folder instead. I still get the same error. I'm a bit out of ideas. Do you have any suggestion where else I could have made a mistake?
Emmanuel Benazera
@beniz
Nov 05 2016 07:21
I'd follow the readme, the models are put into /opt/deepdetect/models by design, you may simply want to override by sharing the volume
ProFire
@ProFire
Nov 05 2016 07:21
I will do that right away.
I will be using this command docker run -d -p 8080:8080 -v /home/profire:/opt/deepdetect/models beniz/deepdetect_cpu
Emmanuel Benazera
@beniz
Nov 05 2016 07:24
then you need to create the covert directory
ProFire
@ProFire
Nov 05 2016 07:25
Doing that right away with mkdir /home/profire/covert
I still get src/io/local_filesys.cc:154: Check failed: allow_null LocalFileSystem: fail to open "/opt/deepdetect/models/covert/0100.model"
Emmanuel Benazera
@beniz
Nov 05 2016 07:28
then log into your container and check that directory
ProFire
@ProFire
Nov 05 2016 07:29
Yes, I see that my files mirror my host folder.
The models folder has permissions drwxr-xr-x 1 dd dd 308 Nov 5 07:25 models
but the models/covert has drwxr-xr-x 1 root root 0 Nov 5 07:25 covert
Could it be due to permissions that is causing the error?
Emmanuel Benazera
@beniz
Nov 05 2016 07:31
I dont know...
ProFire
@ProFire
Nov 05 2016 07:32
I'll give chmod 777 on covert and try again
Hey, no error message in the logs. Looks like it's a permissions matter?
ProFire
@ProFire
Nov 05 2016 07:47
Ok, I successfully run all commands in xgb tutorial. I'm not sure if this is a good question, I want to do number sequence prediction as per my assignment. Do I follow "MLP for single and multi-target regression tasks" in https://deepdetect.com/overview/examples/ ?
Emmanuel Benazera
@beniz
Nov 05 2016 07:52
you can do regression with xgboost and mlp yes
only mlp and other nets support multi-target regression
with xgboost you need a model per target
ProFire
@ProFire
Nov 05 2016 07:58
I sort of understand what you mean. Do you have a tutorial that I can follow that is as close to my assignment as possible? Preferably on xgboost?
It's a very simple 0.1, 0.2, 0.4, 0.8, and next sequence number.
Emmanuel Benazera
@beniz
Nov 05 2016 07:59
not really, you can actually look at xgboost doc, even work with xgboost, the options are the same, they are replicated into the API
there are many xgboost examples out there
ProFire
@ProFire
Nov 05 2016 08:01
Ok, I'll give that a try and play around with the configurations. If I have any questions, I'll be back here. Thanks for the big assist. I hope my questions hasn't been too low-level so far. I'll do my best to read up everything I can.
Emmanuel Benazera
@beniz
Nov 05 2016 08:26
good luck ;)