Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Thi Nguyen
    @duongnt
    @lresende @kevin-bates Hi! Just created this issue elyra-ai/elyra#2119
    Kevin Bates
    @kevin-bates
    Great - thank you Thi!
    Oktay Samur
    @oktaysamr
    image.png
    Hello guys, we are working on Elyra and Apache Airflow integration. We don't want to use cloud system, can we use the Elyra without the upside part of the image?
    Kevin Bates
    @kevin-bates
    Hi @oktaysamr, COS (S3) is currently tightly coupled with the AA components we provide (Notebook, Python/R script). We’re working on custom component enablement which might influence how we manage our built-in components. @akchinSTC or @lresende may be able to offer more insight.
    Alan Chin
    @akchinSTC
    Hi @oktaysamr, Kevin is correct in that our current implementation is tightly couple with S3, which is the case when using any of our current generic components. (which what the above image is showing) We are working on improving data exchange support for operators-> operators for apache airflow, which will support airflow xcoms, avoiding the need to use S3, but thats only the case with airflow provided operators.
    Are you interesting in using some other form of storage e.g. k8s volumes? we can shoot some ideas around in here
    Oktay Samur
    @oktaysamr
    If it is possible @akchinSTC, we can interest k8s volumes and other options without cloud object system. I appreciate it for quick answers @kevin-bates and @akchinSTC.
    Patrick Titzler
    @ptitzler
    To facilitate community communication we are now holding public daily dev meetings. Meeting details can be found at https://elyra.readthedocs.io/en/latest/getting_started/getting-help.html#join-the-daily-development-meetings
    1 reply
    Matheus Hoffmann
    @h0ffmann
    Hi folks, I have a comprehensive question about senstive data inside the pipelines.
    We are using Elyra with KFP and MinIO in production, all services running on GCP.
    One of our concerns is how to handle sensitive data ie: credentials ... Our MinIO version is very attached to KFP, so we dont have much freedom to change it related to IAM/groups etc, so we are using basic auth for it.
    There is a way to use sensitive data inside a pipeline without making it exposed in the cloud object storage?
    One feasible solution we tought is use temporary GCP credentials but I would like to hear if you have a better option.
    Thanks in advance.
    10 replies
    Patrick Titzler
    @ptitzler
    We are trying to wrap up the 3.2 release and plan on publishing on Wednesday, 10/6.
    1 reply
    Kiersten Stokes
    @kiersten-stokes
    Elyra 3.2.0rc1 has just been published. We are hoping to release 3.2.0 officially tomorrow (10/8) after some additional testing
    Kiersten Stokes
    @kiersten-stokes
    Elyra 3.2.0rc2 has just been published. We have moved the 3.2.0 release to early next week to allow time for final testing
    Patrick Titzler
    @ptitzler
    I've assigned Romeo (owner and maintainer) the two component library issues (https://github.com/elyra-ai/component-library/issues) that were opened recently. Unfortunately he's on vacation so we likely won't be able to get a resolution until his return.
    Patrick Titzler
    @ptitzler
    Elyra 3.2.0 is now generally available! A high-level summary of the new features can be found in https://github.com/elyra-ai/elyra/discussions/2191. Learn more about how to get started with custom components for Kubeflow Pipelines in the tutorial -> https://github.com/elyra-ai/examples/tree/master/pipelines/run-pipelines-on-kubeflow-pipelines.
    Patrick Titzler
    @ptitzler
    We will likely publish a 3.2.1 patch release to deliver some authentication related fixes for Kubeflow Pipelines that didn't make the 3.2.0 cut-off.
    Patrick Titzler
    @ptitzler
    Patch 3.2.1 will include an important fix for Kubeflow 1.4 environments
    andre-lx
    @andre-lx

    hey Patrick. I'm trying to install the elyra 3.2.1 from your pr (I really need the fix for the kubeflow 1.4) , but I'm trying to install it with pip like this in the requierements file:

    git+https://github.com/ptitzler/elyra@kf-run-id

    the installation is sucessuful, but when I open the jupyter lab I don't have any UI from elyra. Checking with the CLI I can also see the 3.0.0-dev0 installed.

    Alex Bozarth
    @ajbozarth
    Hi @andre-lx did you run ‘jupyter lab build’ after the pip install? That step is what tells jupyter to install the pip package
    andre-lx
    @andre-lx
    Hi Alex. Yes, I already try that :s let me try once again to check
    andre-lx
    @andre-lx
    Ye, didn't work :s don't understand, installation from the pypi version 3.2.0 works like a charm
    Patrick Titzler
    @ptitzler
    @andre-lx could you paste the output from the validation commands listed in https://elyra.readthedocs.io/en/stable/getting_started/installation.html#verify-the-server-extensions?
    3.2.1 should be available tomorrow (famous last words)
    andre-lx
    @andre-lx

    `

    - Validating...
      elyra  OK
    - Validating...

    /opt/conda/lib/python3.7/site-packages/jupyterlsp/types.py:193: FutureWarning: The alias `()will be deprecated. Usei18n()` instead.
    nodejs = Unicode(help=
    ("path to nodejs executable")).tag(config=True)
    /opt/conda/lib/python3.7/site-packages/jupyterlsp/types.py:195: FutureWarning: The alias `()will be deprecated. Use_i18n()instead. node_roots = List_([], help=_("absolute paths in which to seek node_modules")).tag( /opt/conda/lib/python3.7/site-packages/jupyter_lsp/types.py:200: FutureWarning: The alias_()will be deprecated. Usei18n()` instead.
    [], help=
    ("additional absolute paths to seek node_modules first")
    /opt/conda/lib/python3.7/site-packages/jupyterlsp/manager.py:41: FutureWarning: The alias `()will be deprecated. Usei18n()` instead.
    help=
    ("extra language server specs, keyed by implementation, from conf.d"),
    /opt/conda/lib/python3.7/site-packages/jupyterlsp/manager.py:46: FutureWarning: The alias `()will be deprecated. Usei18n()` instead.
    help=
    ("a dict of language server specs, keyed by implementation"),
    /opt/conda/lib/python3.7/site-packages/jupyterlsp/manager.py:52: FutureWarning: The alias `()will be deprecated. Usei18n()` instead.
    True, help=
    ("try to find known language servers in sys.prefix (and elsewhere)")
    jupyter_lsp 1.4.1 OK

    - Validating...
      jupyter_resource_usage  OK
    - Validating...
      jupyterlab 3.1.7 OK
    - Validating...
      jupyterlab_git 0.32.1 OK
    - Validating...
      nbdime 3.1.0 OK

    config dir: /opt/conda/etc/jupyter
    elyra enabled
    jupyter_lsp enabled
    jupyter_resource_usage enabled
    jupyterlab enabled
    jupyterlab_git enabled
    nbdime enabled
    ```

    well, I will wait for tomorrow than :D
    but the output looks ok right?
    Patrick Titzler
    @ptitzler
    Try "jupyter labextension list"
    it's most meaningful
    should look similar to this

    JupyterLab [version]
    /.../share/jupyter/labextensions
    @jupyter-server/resource-usage [version] enabled OK (python, jupyter-resource-usage)
    @krassowski/jupyterlab-lsp [version] enabled OK (python, jupyterlab_lsp)
    @jupyterlab/git [version] enabled OK (python, jupyterlab-git)

    Other labextensions (built into JupyterLab)
    app dir: /.../share/jupyter/lab
    @elyra/code-snippet-extension [version] enabled OK
    @elyra/metadata-extension [version] enabled OK
    @elyra/pipeline-editor-extension [version] enabled OK
    @elyra/python-editor-extension [version] enabled OK
    @elyra/r-editor-extension [version] enabled OK
    @elyra/theme-extension [version] enabled OK
    nbdime-jupyterlab [version] enabled OK

    Somewhat unrelated, how is you KF/DEX configured? just one provider? We've been having some auth issues with the 3.2.0 release and there's a chance that you might need another fix to get past those (and that fix may or may not be in 3.2.1)
    andre-lx
    @andre-lx
    it's strange, even with a jupyter server extension enable elyra and -> - Writing config: /opt/conda/etc/jupyter
    - Validating elyra...
      elyra  OK

    I can't see elyra in the list of extensions;

    jupyter labextension list
    JupyterLab v3.1.7
    /opt/conda/share/jupyter/labextensions
    nbdime-jupyterlab v2.1.0 enabled OK
    @jupyter-widgets/jupyterlab-manager v3.0.0 enabled OK (python, jupyterlab_widgets)
    @jupyterlab/git v0.32.1 enabled OK (python, jupyterlab-git)
    @krassowski/jupyterlab-lsp v3.9.0 enabled OK (python, jupyterlab-lsp)
    @jupyter-server/resource-usage v0.6.0 enabled OK (python, jupyter-resource-usage)

    Other labextensions (built into JupyterLab)
    app dir: /opt/conda/share/jupyter/lab

    Disabled extensions:
    @jupyterlab/launcher-extension (all plugins)

    The following source extensions are overshadowed by older prebuilt extensions:
    @jupyter-widgets/jupyterlab-manager

    about the istio, are you talking about the istio error connection to the ml-pipeline endpoint?
    Patrick Titzler
    @ptitzler
    Yes
    Are you set up to build Elyra from source? make clean install
    andre-lx
    @andre-lx

    nope, only installing from the repo with: pip install git+https://github.com/ptitzler/elyra@kf-run-id

    And with the pypi packages I never needed anything else, only pip install and use it, and everything worked great till kubeflow 1.4 :P

    Patrick Titzler
    @ptitzler
    Not sure a pip install from a github repo branch will yield the expected results because that might not build/install all dependencies (unlike the make clean install command)
    I just tried your approach and it doesn't work for me either. It's not something we do
    andre-lx
    @andre-lx
    Oh ok then, I was actually expecting that answer ahah I will try the make. But without the istio solved, maybe I need to downgrade to kubeflow 1.3.1 for now, and wait for the fixes. Thank you so much for all the help and for your work
    Patrick Titzler
    @ptitzler
    Just confirmed that the auth fix will be included
    Alan Chin
    @akchinSTC
    Hello!
    Elyra 3.2.1 has just been published to PyPi.
    Change log with bug fixes available here: https://elyra.readthedocs.io/en/stable/getting_started/changelog.html#release-3-2-1-10-18-2021
    Container images to follow shortly...
    andre-lx
    @andre-lx
    Hey guys, trying to install the 3.1.1 version , but I always get this error as soon I open the jupyterlab: Endpoint elyra/schemaspace not found.I already installed the elyra[all]==3.1.1 , and made the jupyterlab build and verify all with the jupyter commands that the extensions are installed and enabled. I have a previous docker image, and that image is working, this is only failing in new builds
    andre-lx
    @andre-lx

    https://github.com/elyra-ai/elyra/compare/v3.1.1...v3.2.1#diff-d0e5e5d4b31372d3dad3a0c68e167de16dd4c10ce9bd340631d4503bacd55c20R42

    looks like, even with the installation of old versions, he pull the server from the most recent versions. In this example the error talks about schemaspace where in version 3.1.1 is namespace

    Kevin Bates
    @kevin-bates
    Hi @andre-lx - I agree with your analysis. Schemaspaces were not introduced until 3.2. This sounds like a packaging/configuration SNAFU with the frontend packages. Had you installed 3.2* prior to this? Perhaps these things are cached somehow, somewhere? Sorry, I don’t know the frontend deployment specifics enough to help here. cc: @ajbozarth
    Patrick Titzler
    @ptitzler
    @andre-lx can you please open an issue? @kevin-bates 3.2.x is causing authentication issues in his environment and we are currently troubleshooting
    Kevin Bates
    @kevin-bates
    :+1:
    andre-lx
    @andre-lx
    yes I will do that later on. Thanks
    andre-lx
    @andre-lx
    issue is open. thanks guys.
    Alex Bozarth
    @ajbozarth
    @andre-lx I just followed up on the issue you opened
    Patrick Titzler
    @ptitzler
    The issue has been resolved for 3.0 and 3.1 releases and patches are now available.