Hydrospheredata/hydro-serving

Daniil Gannota

@danikgan

This is the output after a restart of my laptop and an opening of docker and minikube.

Daniil Gannota

@danikgan

@KineticCookie @tidylobster hi there!

Bulat

@KineticCookie

Hi @danikgan
Sorry for the delay. Didn't get the notification 😕

Could you also send the result of docker logs d26cc60be162 and docker logs 5088293cf1f4

Daniil Gannota

@danikgan

I get a huge output because I ran Hydrosphere too many time, so let me try to send it as a text file

I don't think you'll need all of it, but I am sending anyways just in case

docker logs d26cc60be162.txt

docker logs 5088293cf1f4.txt

Daniil Gannota

@danikgan

@KineticCookie hope this notification finds you!;)

Daniil Gannota

@danikgan

SN Whenever I try to upload a model, it throws me either 504 gateway error or "model build failed" error after line "http://localhost:80 "GET /api/v2/model/version/linear_regression/24 HTTP/1.1" 200 717". I don't really know when it throws 504 and when build failure -- seems unpredictable to me.

Bulat

@KineticCookie

@danikgan hi!

I have just checked your logs

Thread starvation or clock leap detected.

Seems like there are not enough resources for containers and it keeps failing because of connection leaks

Could you try to allocate more resources to the containers and check if it fixes the issue?

Daniil Gannota

@danikgan

What would be the minimal requirements to run hs?

@KineticCookie

Daniil Gannota

@danikgan

I allocated 8GB to Docker and 4 cpus.
I allocated the same for minikube which I use for K8S.

I'll send you the latest logs now, but do you still think it's to do with hs running out of resources?

docker logs d26cc60be162.txt

docker logs 5088293cf1f4.txt

Daniil Gannota

@danikgan

debug: Current cluster: {'cluster': {'server': 'http://localhost'}, 'name': 'local'}
debug: Payload src/ is resolved as /Users/danikgan/Desktop/Hydrosphere/increment_model/src
debug: Payload requirements.txt is resolved as /Users/danikgan/Desktop/Hydrosphere/increment_model/requirements.txt
debug: Popen(['git', 'version'], cwd=/Users/danikgan/Desktop/Hydrosphere/increment_model, universal_newlines=False, shell=None, istream=None)
debug: Popen(['git', 'version'], cwd=/Users/danikgan/Desktop/Hydrosphere/increment_model, universal_newlines=False, shell=None, istream=None)
debug: Error while extracting .git metadata: /Users/danikgan/Desktop/Hydrosphere/increment_model
debug: Can't extract DVC metadata: No module named 'dvc'
Parsed model definition
Name: increment_model
Runtime: hydrosphere/serving-runtime-python-3.6:2.2.1
Install command: pip install -r requirements.txt
Host selector: None
Training data: None
Signature name: increment
Inputs:
| name   | shape   | type     | profile   |
|--------|---------|----------|-----------|
| number | [-1]    | DT_INT32 | NUMERICAL |
Outputs:
| name   | shape   | type     | profile   |
|--------|---------|----------|-----------|
| number | [-1]    | DT_INT32 | NUMERICAL |
Monitoring:
    None
Metadata:
Payload:
    /Users/danikgan/Desktop/Hydrosphere/increment_model/src
    /Users/danikgan/Desktop/Hydrosphere/increment_model/requirements.txt
================================
debug: Creating archive: /Users/danikgan/Desktop/Hydrosphere/increment_model/.hs/increment_model/increment_model.tar.gz
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/increment_model/src as src
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/increment_model/requirements.txt as requirements.txt
Uploading model to http://localhost
================================
debug: MULTIPART POST: http://localhost/api/v2/model/upload. Parts: {'metadata': '{"contract": {"modelName": "model", "predict": {"signatureName": "increment", "inputs": [{"name": "number", "profile": "NUMERICAL", "shape": {"dim": [{"size": -1, "name": ""}], "unknownRank": false}, "dtype": "DT_INT32"}], "outputs": [{"name": "number", "profile": "NUMERICAL", "shape": {"dim": [{"size": -1, "name": ""}], "unknownRank": false}, "dtype": "DT_INT32"}]}}, "hostSelectorName": null, "runtime": {"tag": "2.2.1", "name": "hydrosphere/serving-runtime-python-3.6"}, "name": "increment_model", "installCommand": "pip install -r requirements.txt", "metadata": {}}', 'payload': ('filename', <_io.BufferedReader name='/Users/danikgan/Desktop/Hydrosphere/increment_model/.hs/increment_model/increment_model.tar.gz'>)}
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "POST /api/v2/model/upload HTTP/1.1" 200 650
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "GET /api/v2/model/version/73/logs HTTP/1.1" 200 None
debug: Initialized SSE client from event source <Response [200]>
Build logs:
debug: Dispatching Log event #0, no data...

debug: Dispatching Log event #1, 60 bytes...
Step 1/6 : FROM hydrosphere/serving-runtime-python-3.6:2.2.1
debug: Dispatching Log event #2, 103 bytes...
manifest for hydrosphere/serving-runtime-python-3.6:2.2.1 not found: manifest unknown: manifest unknown
debug: Dispatching EndOfStream event, no data...

================================
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "GET /api/v2/model/version/increment_model/7 HTTP/1.1" 200 702
error: Model build failed

debug: Current cluster: {'cluster': {'server': 'http://localhost'}, 'name': 'local'}
debug: Payload src/ is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/src
debug: Payload requirements.txt is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/requirements.txt
debug: Payload model.h5 is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/model.h5
debug: Popen(['git', 'version'], cwd=/Users/danikgan/Desktop/Hydrosphere/linear_regression, universal_newlines=False, shell=None, istream=None)
debug: Popen(['git', 'version'], cwd=/Users/danikgan/Desktop/Hydrosphere/linear_regression, universal_newlines=False, shell=None, istream=None)
debug: Error while extracting .git metadata: /Users/danikgan/Desktop/Hydrosphere/linear_regression
debug: Can't extract DVC metadata: No module named 'dvc'
Parsed model definition
Name: linear_regression
Runtime: hydrosphere/serving-runtime-python-3.7:2.2.0
Install command: pip install -r requirements.txt
Host selector: None
Training data: None
Signature name: infer
Inputs:
| name   | shape   | type      | profile   |
|--------|---------|-----------|-----------|
| x      | [-1, 2] | DT_DOUBLE | NUMERICAL |
Outputs:
| name   | shape   | type      | profile   |
|--------|---------|-----------|-----------|
| y      | [-1]    | DT_DOUBLE | NUMERICAL |
Monitoring:
    None
Metadata:
Payload:
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/src
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/requirements.txt
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/model.h5
================================
debug: Creating archive: /Users/danikgan/Desktop/Hydrosphere/linear_regression/.hs/linear_regression/linear_regression.tar.gz
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/src as src
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/requirements.txt as requirements.txt
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/model.h5 as model.h5
Uploading model to http://localhost
================================
debug: MULTIPART POST: http://localhost/api/v2/model/upload. Parts: {'metadata': '{"contract": {"modelName": "model", "predict": {"signatureName": "infer", "inputs": [{"name": "x", "profile": "NUMERICAL", "shape": {"dim": [{"size": -1, "name": ""}, {"size": 2, "name": ""}], "unknownRank": false}, "dtype": "DT_DOUBLE"}], "outputs": [{"name": "y", "profile": "NUMERICAL", "shape": {"dim": [{"size": -1, "name": ""}], "unknownRank": false}, "dtype": "DT_DOUBLE"}]}}, "hostSelectorName": null, "runtime": {"tag": "2.2.0", "name": "hydrosphere/serving-runtime-python-3.7"}, "name": "linear_regression", "installCommand": "pip install -r requirements.txt", "metadata": {}}', 'payload': ('filename', <_io.BufferedReader name='/Users/danikgan/Desktop/Hydrosphere/linear_regression/.hs/linear_regression/linear_regression.tar.gz'>)}
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "POST /api/v2/model/upload HTTP/1.1" 200 665
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "GET /api/v2/model/version/74/logs HTTP/1.1" 200 None
debug: Initialized SSE client from event source <Response [200]>
Build logs:
debug: Dispatching Log event #0, no data...

debug: Dispatching Log event #1, 60 bytes...
Step 1/6 : FROM hydrosphere/serving-runtime-python-3.7:2.2.0
debug: Dispatching Log event #2, 103 bytes...
manifest for hydrosphere/serving-runtime-python-3.7:2.2.0 not found: manifest unknown: manifest unknown
debug: Dispatching EndOfStream event, no data...

================================
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "GET /api/v2/model/version/linear_regression/35 HTTP/1.1" 200 717
error: Model build failed

Even trying to upload different ML models produces exactly the same procedure for error

Bulat

@KineticCookie

Hi!
Seems like a problem here is that docker can't find the image.
I re-released the images, could you try again?

Daniil Gannota

@danikgan

Okay, so I deleted the old hs images, installed the new ones but still the same error

Could it be because I am on Helm 3? I need to find how to downgrade to v2.9 then

Dmitriy Kharlamov

@akastav

Hi Daniil! Could you try this command? docker pull hydrosphere/serving-runtime-python-3.7:2.2.0

Daniil Gannota

@danikgan

2.2.0: Pulling from hydrosphere/serving-runtime-python-3.7
Digest: sha256:e62e47b3352b23788e4e31f3e4a5636328a17cc65e7d44b5e234a6f3868cd977
Status: Image is up to date for hydrosphere/serving-runtime-python-3.7:2.2.0
docker.io/hydrosphere/serving-runtime-python-3.7:2.2.0

debug: Dispatching Log event #406, 10 bytes...
Extracting
debug: Dispatching Log event #407, 13 bytes...
Pull complete
debug: Dispatching Log event #408, 79 bytes...
Digest: sha256:e62e47b3352b23788e4e31f3e4a5636328a17cc65e7d44b5e234a6f3868cd977
debug: Dispatching Log event #409, 79 bytes...
Status: Downloaded newer image for hydrosphere/serving-runtime-python-3.7:2.2.0
debug: Dispatching Log event #410, 17 bytes...
---> 90f743fd43f9
debug: Dispatching Log event #411, 45 bytes...
Step 2/6 : LABEL MODEL_NAME=linear_regression
debug: Dispatching Log event #412, 28 bytes...
---> Running in 1373c319bfc5
debug: Dispatching Log event #413, 44 bytes...
Removing intermediate container 1373c319bfc5
debug: Dispatching Log event #414, 17 bytes...
---> dc43c044b0a8
debug: Dispatching Log event #415, 32 bytes...
Step 3/6 : LABEL MODEL_VERSION=2
debug: Dispatching Log event #416, 28 bytes...
---> Running in 34b07afa09bb
debug: Dispatching Log event #417, 44 bytes...
Removing intermediate container 34b07afa09bb
debug: Dispatching Log event #418, 17 bytes...
---> 8d29ff4fe4dc
debug: Dispatching Log event #419, 27 bytes...
Step 4/6 : ADD model /model
debug: Dispatching Log event #420, 17 bytes...
---> bdeb5bca1ddb
debug: Dispatching Log event #421, 31 bytes...
Step 5/6 : WORKDIR /model/files
debug: Dispatching Log event #422, 28 bytes...
---> Running in 9aaea0b13b3f
debug: Dispatching Log event #423, 44 bytes...
Removing intermediate container 9aaea0b13b3f
debug: Dispatching Log event #424, 17 bytes...
---> a86bfaf28563
debug: Dispatching Log event #425, 46 bytes...
Step 6/6 : RUN pip install -r requirements.txt
debug: Dispatching Log event #426, 28 bytes...
---> Running in 0aa6ee58f721
debug: Dispatching message event, no data...

debug: Dispatching message event, no data...

debug: Dispatching Log event #427, 332 bytes...
[91mWARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fd4ef890a50>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/keras/

debug: Dispatching message event, no data...

debug: Dispatching message event, no data...

debug: Dispatching Log event #428, 332 bytes...
[91mWARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fd4ef8a7490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/keras/

debug: Dispatching message event, no data...

debug: Dispatching message event, no data...

debug: Dispatching Log event #429, 332 bytes...
[91mWARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fd4ef8a7590>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/keras/

debug: Dispatching message event, no data...

debug: Dispatching message event, no data...

debug: Dispatching Log event #430, 332 bytes...
[91mWARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fd4ef8a7c90>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/keras/

debug: Dispatching message event, no data...

debug: Dispatching message event, no data...

debug: Dispatching Log event #431, 332 bytes...
[91mWARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fd4ef8a7dd0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/keras/

debug: Dispatching message event, no data.

This is what I get at the end of hs -v upload

@akastav @KineticCookie

Bulat

@KineticCookie

🤔
Seems like it can't install dependency from requirements.txt

Do you have some specific network settings, which could deny egress traffic from contrainers? Or you installed hydroserving from docker-compose.yaml as-is?
People encountered similar issues, and seems like it's related to dns lookup on a host with docker daemon: https://forums.docker.com/t/facing-issue-with-creating-angular-application-image-using-docker/87270

You could troubleshoot dns lookup from within a container in your machine. If the issue still persists after it, we'll try to come up with some fix/workaround.

Daniil Gannota

@danikgan

Yes, I do indeed have a custom network setting. If it can't install the dependencies, might this mean an issue with pip?

And yes, I installed hydro serving from docker-compose.yaml as is. I tried to configure it to use my pip setting: under docker-compose.yaml I have this line:

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./pip:/root/.pip

Under .pip I have the correct pip.conf and certificate from my workplace.

But when I run docker-compose up and connect to the shell of manager container, I find that hydrosphere does not have pip installed.

Daniil Gannota

@danikgan

@KineticCookie @akastav

Ilnur Garifullin

@tidylobster

@danikgan You're on a right track, but you don't need to manipulate docker-compose.yaml to deliver your pip.conf file to the build context. One way of doing that might be just to include that file in the serving.yamlfile's payload and update install-comand (which is just a shell script). If we're talking about our linear_regression example, then it will look like this:

kind: Model
name: linear_regression
runtime: "hydrosphere/serving-runtime-python-3.6:2.2.1"
install-command: "PIP_CONFIG_FILE=pip.conf pip install -r requirements.txt"
payload:
  - "src/"
  - "requirements.txt"
  - "pip.conf" # specify location of your pip.conf 
  - "model.h5"

contract:
  name: infer
  inputs:
    x:
      shape: [-1, 2]
      type: double
      profile: numerical
  outputs:
    y:
      shape: [-1]
      type: double
      profile: numerical

In this case I've just specified that pip.conf file lies within my current build context. So, pip should utilize pip.conf and you won't see Retrying (Retry(total=1, connect=None... during hs -v upload.

If you don't want to put pip.conf into payload of your model, you can update install-command pip install -r requirements.txt to include all network settings required for your environment. For instance, pip install --proxy <proxy> -r requirements.txt.

Daniil Gannota

@danikgan

Alright, we're on the right track -- that's great! Here's what I tried doing:

Edited the serving.yaml file as specified.
Added a certificate to the payload (as I need it for pip)
Specified the location of the certificate under pip.conf file as cert = terra_bin.cert
Turned on my VPN as it requires access to my workplace internet

I still get an error though (and I tried various combinations I could think of to fix this). Could it be an issue with accepting the certificate? The pip.conf file should be correct as this is the same one I am using for my own local pip configuration (which works btw).

Here's the output I get:

debug: Current cluster: {'cluster': {'server': 'http://localhost'}, 'name': 'local'}
debug: Payload src/ is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/src
debug: Payload requirements.txt is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/requirements.txt
debug: Payload pip.conf is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/pip.conf
debug: Payload tecca_bin.cer is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/tecca_bin.cer
debug: Payload model.h5 is resolved as /Users/danikgan/Desktop/Hydrosphere/linear_regression/model.h5
debug: Popen(['git', 'version'], cwd=/Users/danikgan/Desktop/Hydrosphere/linear_regression, universal_newlines=False, shell=None, istream=None)
debug: Popen(['git', 'version'], cwd=/Users/danikgan/Desktop/Hydrosphere/linear_regression, universal_newlines=False, shell=None, istream=None)
debug: Error while extracting .git metadata: /Users/danikgan/Desktop/Hydrosphere/linear_regression
debug: Can't extract DVC metadata: No module named 'dvc'
Parsed model definition
Name: linear_regression
Runtime: hydrosphere/serving-runtime-python-3.7:2.2.0
Install command: PIP_CONFIG_FILE=pip.conf pip install -r requirements.txt
Host selector: None
Training data: None
Signature name: infer
Inputs:
| name   | shape   | type      | profile   |
|--------|---------|-----------|-----------|
| x      | [-1, 2] | DT_DOUBLE | NUMERICAL |
Outputs:
| name   | shape   | type      | profile   |
|--------|---------|-----------|-----------|
| y      | [-1]    | DT_DOUBLE | NUMERICAL |
Monitoring:
    None
Metadata:
Payload:
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/src
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/requirements.txt
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/pip.conf
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/tecca_bin.cer
    /Users/danikgan/Desktop/Hydrosphere/linear_regression/model.h5
================================
debug: Creating archive: /Users/danikgan/Desktop/Hydrosphere/linear_regression/.hs/linear_regression/linear_regression.tar.gz
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/src as src
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/requirements.txt as requirements.txt
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/pip.conf as pip.conf
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/tecca_bin.cer as tecca_bin.cer
debug: Archiving /Users/danikgan/Desktop/Hydrosphere/linear_regression/model.h5 as model.h5
Uploading model to http://localhost
================================
debug: MULTIPART POST: http://localhost/api/v2/model/upload. Parts: {'metadata': '{"contract": {"modelName": "model", "predict": {"signatureName": "infer", "inputs": [{"name": "x", "profile": "NUMERICAL", "shape": {"dim": [{"size": -1, "name": ""}, {"size": 2, "name": ""}], "unknownRank": false}, "dtype": "DT_DOUBLE"}], "outputs": [{"name": "y", "profile": "NUMERICAL", "shape": {"dim": [{"size": -1, "name": ""}], "unknownRank": false}, "dtype": "DT_DOUBLE"}]}}, "hostSelectorName": null, "runtime": {"tag": "2.2.0", "name": "hydrosphere/serving-runtime-python-3.7"}, "name": "linear_regression", "installCommand": "PIP_CONFIG_FILE=pip.conf pip install -r requirements.txt", "metadata": {}}', 'payload': ('filename', <_io.BufferedReader name='/Users/danikgan/Desktop/Hydrosphere/linear_regression/.hs/linear_regression/linear_regression.tar.gz'>)}
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "POST /api/v2/model/upload HTTP/1.1" 200 690
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "GET /api/v2/model/version/11/logs HTTP/1.1" 504 173
error: Got server error <Response [504]>
Error: Server returned an error: <html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>openresty/1.15.8.3</center>
</body>
</html>

I get the output above for tag 2.2.0. For tag 2.2.1 I get this ending:

debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "POST /api/v2/model/upload HTTP/1.1" 200 690
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "GET /api/v2/model/version/12/logs HTTP/1.1" 404 74
warning: Build logs are not available
debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "GET /api/v2/model/version/linear_regression/12 HTTP/1.1" 200 742
error: Model build failed
{"model": {"id": 1, "name": "linear_regression"}, "image": {"name": "linear_regression", "tag": "12"}, "installCommand": "PIP_CONFIG_FILE=pip.conf pip install -r requirements.txt", "finished": "2020-06-03T11:25:24.405Z", "modelContract": {"modelName": "model", "predict": {"signatureName": "infer", "inputs": [{"profile": "NUMERICAL", "dtype": "DT_DOUBLE", "name": "x", "shape": {"dim": [{"size": -1, "name": ""}, {"size": 2, "name": ""}], "unknownRank": false}}], "outputs": [{"profile": "NUMERICAL", "dtype": "DT_DOUBLE", "name": "y", "shape": {"dim": [{"size": -1, "name": ""}], "unknownRank": false}}]}}, "id": 12, "status": "Failed", "metadata": {}, "kind": "Internal", "modelVersion": 12, "runtime": {"name": "hydrosphere/serving-runtime-python-3.7", "tag": "2.2.1"}, "created": "2020-06-03T11:25:19.746Z"}

Same outputs go for hydrosphere/serving-runtime-python-3.6, respectively.

Any thoughts? @tidylobster @KineticCookie @akastav

Ilnur Garifullin

@tidylobster

Hi @danikgan
Sorry for late reply. What path are you using in your pip.conf to point to your certificate?

Ilnur Garifullin

@tidylobster

If you place both pip.conf and your certificate (let's say cert.pem) into the payload within serving.yaml, they will be available under /model/files/pip.conf, /model/files/cert.pem paths. So, you might try to update your pip.conf to use the correct path to your cert.pem.

Daniil Gannota

@danikgan

Unfortunately, this didn't help:

I change the pip config file to cert=/model/files/cert.pem
Launched VPN and all the rest is the same as described above

Shall I provide you with some logs or additional context?
@tidylobster @KineticCookie @akastav

This is my current error:

debug: Starting new HTTP connection (1): localhost:80
debug: http://localhost:80 "POST /api/v2/model/upload HTTP/1.1" 504 173
error: Got server error <Response [504]>
Error: Server returned an error: <html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>openresty/1.15.8.3</center>
</body>
</html>

Ilnur Garifullin

@tidylobster

I'm not sure why we are back on 504 error. The server didn't even start building the model. Can this be because of low resources, as @KineticCookie mentioned here? https://gitter.im/Hydrospheredata/hydro-serving?at=5ec7c91ef0b8a2053aa80284

Ilnur Garifullin

@tidylobster

So, did you change your settings for docker daemon from last try?

Daniil Gannota

@danikgan

You're right!:) Finally, the sacred words of Success: {"model": {...

Ilnur Garifullin

@tidylobster

Great! I'm glad it worked.

gaopan1996

@gaopan1996

Hi, all. I have started the quickstart documentation for the hs cli on my Windows10 machine. I installed hs==2.4.0 and am getting this error

hs --version
Traceback (most recent call last):
    File "c:\program files\python36\lib\runpy.py", line 193, in _run_module_as_main
        "__main__", mod_spec)
    File "c:\program files\python36\lib\runpy.py", line 85, in _run_code
        exec(code, run_globals)
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\Scripts\hs.exe\__main__.py", line 5, in <module>
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydroserving\cli\commands\__init__.py", line 1, in <module>
        from hydroserving.cli.commands.hs import * 
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydroserving\cli\commands\hs.py", line 7, in <module>
        from hydroserving.cli.context_object import ContextObject
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydroserving\cli\context_object.py", line 7, in <module>
        from hydroserving.core.apply import ApplyService
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydroserving\core\apply.py", line 8, in <module>
        from hydroserving.core.deployment_config.parser import parse_deployment_configuration
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydroserving\core\deployment_config\parser.py", line 3, in <module>
        from hydrosdk import DeploymentConfiguration
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydrosdk\__init__.py", line 3, in <module>
        from hydrosdk.modelversion import LocalModel, ModelVersion
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydrosdk\modelversion.py", line 19, in <module>
        from hydrosdk.contract import ModelContract_to_contract_dict, contract_dict_to_ModelContract, validate_contract
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydrosdk\contract.py", line 11, in <module>
        from hydrosdk.data.types import alias_to_proto_dtype, shape_to_proto, PY_TO_DTYPE, np_to_proto_dtype, proto_to_np_dtype
    File "C:\Users\K27793\AppData\Roaming\Python\Python36\site-packages\hydrosdk\data\types.py", line 130, in <module>
    np.float128: None,
AttributeError: module 'numpy' has no attribute 'float128'

which is related to this issue winpython/winpython#613.
Seems like an issue with hydro-sdk on this line https://github.com/Hydrospheredata/hydro-serving-sdk/blob/master/hydrosdk/data/types.py#L130

Downgrading to hs==2.2.0, which doesn't have hydro-sdk dependency, worked on Windows10.

Yuriy

@Valenzione

Hi @gaopan1996 , thanks for noticing this issue! We indeed do not perform multi-platform testing of our client libraries, so we are mostly unaware of possible issues for our Windows users.

Since this is an issue, I suggest you posting the same text you’ve sent us here to our Github issues via this link: https://github.com/Hydrospheredata/hydro-serving/issues/new

I'll return shortly after with a solution and post it directly to the issue you will have created.

gaopan1996

@gaopan1996

Thanks @Valenzione. Created issue here Hydrospheredata/hydro-serving#330

Where communities thrive

People

Repo info

Activity