beniz/deepdetect

Deep Learning API and Server

programmeddeath1

@programmeddeath1

I'v actually run the git pull and subsequent steps through my root user. im getting " host not found in upstream "jupyter:8888" in /etc/nginx/nginx.conf:50
nginx: [emerg] host not found in upstream "jupyter:8888" in /etc/nginx/nginx.conf:50"

Iv found this in jolibrain/dd_platform_docker#1 to be possibly because of running through root user

Emmanuel Benazera

@beniz

OK, the jupyter notebook docker is sketchy because of the internal user. Definitely try not running the containers as root.

FYI, the user rights are at play when accessing the host's drive.

programmeddeath1

@programmeddeath1

Yes added the user to docker group and ran is working now. Thank you :)

Can I add my own models and codebase? or is it restricted to the existing models?

Emmanuel Benazera

@beniz

there's no restriction, but new models need to either have been trained on the platform, or brought in and made "compatible"... This last step depends on what backend you are using.

programmeddeath1

@programmeddeath1

i would like to add deepfigures-open and pdffigures2 from allenai's repo as new services for an NLP task. Is that possible?

https://github.com/allenai/deepfigures-open

Emmanuel Benazera

@beniz

not easily at all

AJAY SAHU

@ajay01994

https://github.com/allenai/allennlp

same here

what can we do ? to add these

Emmanuel Benazera

@beniz

The easiest way would be to use these tools to build a dataset, and train an object detector with the platform. You'd end up with your own tables, etc... detection model

you'd need to convert PDF pages into images beforehand.

also this looks like a trivial task, you probably don't need a ResNet-101 to do that...

AJAY SAHU

@ajay01994

basic idea is to do PDF processing using the above ML repo and take out texts ,images vectors ,B box ,metadata for PDF breakdown to do further text processing and NLP tasks on the above

like in a pipeline

what do you suggest @beniz

all pdf would need to be converted into images ?

there are scanned pdf as well . so then it would require to run sandwitch lib to amke it OCR ready

*make

Emmanuel Benazera

@beniz

if you look at Figure 3 of their paper, you see they actually use standard object detection from images

they appear to provide the dataset as well, so you should be able to train your own figure detector.

AJAY SAHU

@ajay01994

https://github.com/allenai/pdffigures2

Emmanuel Benazera

@beniz

unless you know what you are doing, I'd use the allenlp code directly

AJAY SAHU

@ajay01994

yes that makes sense , was thinking to create PDF processing as a service

& so for text generation can we add - https://github.com/minimaxir/gpt-2-simple

https://blog.usejournal.com/opengpt-2-we-replicated-gpt-2-because-you-can-too-45e34e6d36dc

AJAY SAHU

@ajay01994

https://colab.research.google.com/drive/1esbpDOorf7DQJV8GXWON24c-EQrSKOit

AJAY SAHU

@ajay01994

if you guys help us in doing that , it would be wonderful since i am quite new to deep detect

Emmanuel Benazera

@beniz

Interesting project, indeed! We're quite busy at the moment though :) You'll sort it out, there's a lot of code ready to use with gpt-2 and allenai stuff.

AJAY SAHU

@ajay01994

Sure !

we will try :)

W8TERM3LON

@W8TERM3LON

Every time I try to install the gpu version it displays
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.deepdetect: 'runtime'
Unsupported config option for services.gpustat_server: 'runtime'

does anybody know why it's unable to install?

W8TERM3LON

@W8TERM3LON

ok I figured it out. It was a out of date docker-compose.

W8TERM3LON

@W8TERM3LON

Now whenever I try to train a model this shows up

Alexandre Girard

@alx

Hi @W8TERM3LON I suppose you're using the example on this page ? https://www.deepdetect.com/platform/docs/image-classification/

in the Classification options, you should replace host and port with the following values:

host="localhost",
port=1912,
path="/api/deepdetect",

thanks for the feedback, we'll correct the platform documentation on the website

Emmanuel Benazera

@beniz

@cchadowitz-pf I had the basic CI run on your PR, and I'll run another test by hand soon when I can before merging. That's if you don't mind the delay. I appreciate your patience.

cchadowitz-pf

@cchadowitz-pf

that's awesome, thanks! not a problem. any issues come up yet?

(you're talking about #657?)

Emmanuel Benazera

@beniz

Yes #657, the CI had failed due to lack of space. Will run the dlib build by hand then add dlib to CI quickly after that.

cchadowitz-pf

@cchadowitz-pf

:+1:

Where communities thrive

People

Repo info

Activity