Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    zidev25
    @zidev25
    okay
    done
    Ilnur Garifullin
    @tidylobster
    Add a cluster and try to upload again
    zidev25
    @zidev25
    did that - same error
    Is there a stable version of the platform that is available ? That we have been using let us say 5 days back. How do we get to that?
    I don't need the cutting edge - just a stable version that I can deploy some models and do some scaling testing.
    I tried the release .tar.gz as suggested in the documentation - 2.0.1 version did not work ..
    Konstantin Makarychev
    @mkf-simpson
    Hi @zidev25
    Could you please share your port-forwardcommand? Try to open http://localhost:8080 in a browser: can you see hydro-serving UI? Also k8s resource list can help: kubectl get all -n <your-namespace>
    MATHANAIHT
    @MATHANAIHT
    Hi i am new to hydro serving, facing same issue. i have Kubernetes cluster with istio metrics injected. After enabled istio then i installed hydro serving, in this situation "hs upload" getting request failed. Note : serving-2.0.0 and hs-version 2.0.0.
    Konstantin Makarychev
    @mkf-simpson
    @MATHANAIHT try to update serving and hs to 2.0.4. There are some fixes that could be related to your issue. If something will go wrong, add -v flag to upload command: hs -v upload for more details.
    MATHANAIHT
    @MATHANAIHT
    serving-ui.png
    Thanks for the reply. i have updated serving and hs to 2.0.4. Still the problem exist. Please find my logs on serving-ui and hs -v upload
    hs -v upload.png
    ui.png
    MATHANAIHT
    @MATHANAIHT

    helm ls
    serving 1 Wed Apr 24 15:15:53 2019 DEPLOYED serving-2.0.4 default

    hs --version
    hs version 2.0.4

    Bulat
    @KineticCookie
    hi @MATHANAIHT
    could please share the result of hs cluster buildinfo command
    MATHANAIHT
    @MATHANAIHT
    hs cluster buildinfo.png
    MATHANAIHT
    @MATHANAIHT

    Hydro serving installation and hs upload is working fine on 2.0.4 without istio.

    if istio installed before hydro serving then hs upload is not working.

    i solved hs upload issue with following steps

    1. Inatall hydro serving and hs v2.0.4

    2. Install istio v1.1.3

    3. Port forward serving-ui

    4. hs upload.

    if pods created before istio installation, must delete the running pods. Because it doesn't have the istio sidecar injection.

    zidev25
    @zidev25
    I have a question about HS_INSTANCE_NAME=ner-1 - when we describe a pod of an application. Is there a reason to not use app=ner-1
    Konstantin Makarychev
    @mkf-simpson
    We use the same code to mark containers for kubernetes (annotations) and docker-compose (labels). Abstractions, you know :)
    zidev25
    @zidev25
    that is the way to go
    I agree
    just curious - will adding another label that is more commonly used change the internal flow dynamic?
    app=ner-1
    especially when debugging - with different cloud providers - what is common is easier for them to relate to - while we have our own labels for the internal flow dynamic
    Konstantin Makarychev
    @mkf-simpson
    Everything will work as it should until you change HS_* markers. We rely only on them
    zidev25
    @zidev25
    ok
    thanks @mkf-simpson - one way is to add it post deplyment of the application - at the service level - or it could become part of the code base generating the deployment/service
    Will try the post deployment updating the service.yaml file and will let you know
    Konstantin Makarychev
    @mkf-simpson
    We'll think how users could configure k8s deployments, and will include solution in the roadmap
    zidev25
    @zidev25
    ok thanks @mkf-simpson
    Amey Agrawal
    @AgrawalAmey
    Hi, I am trying to get familiar with hydro-serving. I deployed an application with Increment model described in the documentation. The endpoint is working fine but I am not able to use monitoring and profiling features.
    On loading the monitoring page, a pop up shows up with error Error: Error: Can't fetch data from monitoring service
    When I try to add new metric it shows error Error: status: 502, error: HttpErrorResponse, message: Http failure response for http://localhost:20000/monitoring/metricspec: 502 Bad Gateway
    Ilnur Garifullin
    @tidylobster
    Hi @AgrawalAmey.
    Sorry for the late reply. Monitoring service is not available as OSS right now, it's an enterprise feature. If you want, we can make a demo for you showing off monitoring service with various metrics, data profiling, reqstore. Please, submit a request via site https://hydrosphere.io/ for a more detailed discussion.
    MATHANAIHT
    @MATHANAIHT
    Screenshot from 2019-05-22 15-17-07.png
    Hi, i have installed hydro serving on Kubernetes cluster with version 2.0.4. i am trying to upload model, model getting to upload failed. i checked log on the manger pod above error is listed.
    Chaitra1996
    @Chaitra1996
    image.png
    Hii, i have installed hydro serving on docker cluster with version 2.0.4. when am trying to upload model, am getting model build failed error. Could someone help me out with this issue.
    Ilnur Garifullin
    @tidylobster
    Hi @Chaitra1996 , can you perform hs -v upload for more verbose logs?
    Chaitra1996
    @Chaitra1996
    image.png
    Hi @tidylobster i tried the command that u asked for. I am able to see the verbose logs. As shown in the image i tried to resolve the issue by setting the environment variable for git executable. But, still the error continues.
    Ilnur Garifullin
    @tidylobster
    Well, hs just tries to parse .git and .dvc metadata, it's not essential for the model upload. Can you share full logs of the model upload? What is happening when the archive is packed and is being uploaded to the serving?
    Bulat
    @KineticCookie

    Hi @Chaitra1996
    This error occured because manager service encountered some issue while processing you model files. To dig deeper into it, we'll need to look for manager service logs - they should contain progress of model processing.

    In addition to that, in the latest dev version we implemented logs passthrough, so they'll be available to you via CLI or web UI. We'll release it as soon as we stabilize all the services. Looking forward to notify you when it's done! :)

    Chaitra1996
    @Chaitra1996
    image.png
    image.png
    @tidylobster the above two images are the logs that have appeared after running the upload command. And its not getting uploaded on to serving.
    Chaitra1996
    @Chaitra1996
    Hi @KineticCookie , in the current version we will not be able to look for the logs?
    Bulat
    @KineticCookie
    @Chaitra1996 you can look for the container/pod logs, but it would require using kubectl.
    And yes, logs API (if I can call that an API) unfortunately is not in the 2.0.4 release. :(
    Chaitra1996
    @Chaitra1996
    @KineticCookie Thank you