{
"httpCode": 500,
"error": "Binding context MLDB::SqlExpressionMldbScope does not support bound parameters ($1... or $name)"
}
"httpCode": 500,
"error": "Binding context MLDB::SqlExpressionMldbScope does not support bound parameters ($1... or $name)"
}
mldb.put("/v1/functions/nearestneighbors", {
"type": "sql.query",
"params": {
"query": "SELECT * FROM transpose((SELECT nearest" + collection + "({coords: inception({url: $URL})})[distances] AS *))",
"output": "FIRST_ROW"
}
})
"type": "sql.query",
"params": {
"query": "SELECT * FROM transpose((SELECT nearest" + collection + "({coords: inception({url: $URL})})[distances] AS *))",
"output": "FIRST_ROW"
}
})
I want to get transposed output of embedding dataset that I get from embedding.neighbors type function
so It’s actually a known issue with parameters not making into the subselect in an sql.query. the best workaround I can suggest is to change your calls to your
nearestneighbors function to a call to the /v1/query and put the complete sql call with your SELECT * FROM tranapose() and adding a LIMIT 1 to get essentially the same output as you would with sql.query
question about embedding dataset: is there a way to calculate how much approximately embedding dataset will take RAM? For example, using your inception model with second to last layer (~2k per instance) features for knn calculation for 10000 images RAM consumption increases by about 2.5GB, after this code:
mldb.put("/v1/procedures/embedder", {
"type": "transform",
"params": {
"inputData": """
SELECT inception({url: URL}) AS *
FROM images_tmp LIMIT 10000
""",
"outputDataset": {
"id": "embeddedimages%s" % collection,
"type": "embedding"
}
}
})
"type": "transform",
"params": {
"inputData": """
SELECT inception({url: URL}) AS *
FROM images_tmp LIMIT 10000
""",
"outputDataset": {
"id": "embeddedimages%s" % collection,
"type": "embedding"
}
}
})
so I guess for 100ks of images, this kind of embedding would require a huge RAM for knn application and there is no way to do it more efficiently?
@GedasZ sorry for not answering ealier. There’s no way to ask MLDB how much a resulting operation would take. However things should be at worst linear so you can infer the size from doing the operation on a few datapoints. The increase of ram you mention seems bigger than it should be. we’ll make some checks on our side and I’ll let you know
hey @robomotic. potentially you want a combination of both functions and procedures. functions can be used to take one of many columns in a row and return one to many columns back, while a procedure can take the result of a whole query and return a new dataset or any other artefact. as for native vs plugin, I’d go with plugin. we have a sample plugin you can base your code on here: https://github.com/mldbai/mldb_sample_plugin
Hello... We need to use Pandas version 0.18 or greater (maybe via experimental.external procedure). Is there a way to do so? My assumption was that experimental.external would use the global python interpreter and I pip installed pandas 0.19 but the plugin keeps throwing errors relating to pandas 0.17.
apparently it cannot be found in the community repository
so I am in community's AMI and that strings returns nothing
maybe is only in some regions?
I don't think I can add a screenshot but basically it says not found in 4 regions
well mistery solved it seems to be available only in the north american region
I just double checked and it is available in tokyo, seoul, oregon, n. cal
make sure you have proper security group rules in place if you do so.
so in other words I can only rely on network authentication to the host but can't filter at the application level?
@halidm It is possible to run MLDB on an older server, but you will need to recompile it and modify the Makefile here https://github.com/mldbai/mldb/blob/master/ext/tensorflow.mk#L397 to a) add a sse41 mode, and b) set up the compile flags just below for that
As for MLDB exiting, I'd say that you have system or docker limits set up in such a way that it runs out of memory.
Is anyone had this:
Aborting boot: directory mapped to /mldb_data owned by root
Expected owner uid from MLDB_IDS is: 1003
If the directory mapped to /mldb_data did not exist before launching MLDB,
it is automatically created and owned by root. If this is what happened,
please change the owner of the directory and relaunch.
Hello guys,
So I have a question to ask. I trained a model using inception from tensorflow (to be able to classify food from images), at the end I have a graph file (.pb) and I want to use that graph to predict food from images but I want to make it in an API, how can I do that exactly? I saw the example from mldb where they used inception model as it is to predict stuff (they had a url of inception model and they pointed to the graph file in the inception folder) but I want to use my graph to predict food which is a re-trained model. How can I make that into an API that can be used by let's say websites to predict images? I am also thinking about the graph file and will that affect my API, I mean, the graph file is relatively big (~80Mb) so will this affect my API since this API needs to use the graph each time it wants to predict a food from image? I am sorry if this question is silly, I am not really a software engineer, I do computer vision stuff and I need to make this API as part of an assignment.
Any help is greatly appreciated :)
Thanks
So I have a question to ask. I trained a model using inception from tensorflow (to be able to classify food from images), at the end I have a graph file (.pb) and I want to use that graph to predict food from images but I want to make it in an API, how can I do that exactly? I saw the example from mldb where they used inception model as it is to predict stuff (they had a url of inception model and they pointed to the graph file in the inception folder) but I want to use my graph to predict food which is a re-trained model. How can I make that into an API that can be used by let's say websites to predict images? I am also thinking about the graph file and will that affect my API, I mean, the graph file is relatively big (~80Mb) so will this affect my API since this API needs to use the graph each time it wants to predict a food from image? I am sorry if this question is silly, I am not really a software engineer, I do computer vision stuff and I need to make this API as part of an assignment.
Any help is greatly appreciated :)
Thanks
It's very similar. The easiest way is to include the same structure as in the example here: https://docs.mldb.ai/doc/#/v1/plugins/tensorflow/doc/TensorflowGraph.md.html create a zip file with the model.pb and a text file with the label names, and just change the name of the file to match the path of the zip file and the other filenames to match those inside. The graph will only be loaded once, and then will stay in memory every time the API is called. If you follow that example, you can get a prediction out by calling the /v1/query route just like in that example. Note that making an API is something that you kind of need to be a developer for, as you need to know about REST and URL encoding and all that stuff.