Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Dannon
    @dannon
    Ouch. Bad timing for us to move to .com
    I do see:
    We will be offering an allotment of OSS minutes that will be reviewed and allocated on a case by case basis. Should you want to apply for these credits please open a request with Travis CI support stating that you’d like to be considered for the OSS allotment. Please include:

    Your account name and VCS provider (like travis-ci.com/github/[your account name] )
    How many credits (build minutes) you’d like to request (should your run out of credits again you can repeat the process to request more or discuss a renewable amount)
    So if anyone has a contact there, we can leverage that, or I (or someone else) can open a support request.
    Pablo Moreno
    @pcm32
    Yes, I just did that for our EBI repos, maybe better if someone else says pretty please from the Galaxy side.
    Apparently they were getting a lot of bitcoin mining and related stuff.
    Dannon
    @dannon
    Ok, I'll send an email. JBrowse folks just had all their builds fail for the same reason, heh.
    robertbeudeker
    @robertbeudeker
    Hi All, I'm new to the community. Hope it's not a silly question. I'm currently looking at using AzureAD als OpenID IDP for galaxy. I looked at: https://galaxyproject.org/authnz/config/oidc/ currently there are 6 custom providers AzureAD is not one of them. Is there any change AzureAD is added to the list or maybe an other approach that i can take?
    Nuwan Goonasekera
    @nuwang
    Hi @robertbeudeker We think keycloak is a good option here and is something we use in practice to integrate with external identity providers (e.g. in the GVL). We connect Galaxy to KeyCloak via OIDC, and KeyCloak can in turn be connected to AzureAD.
    robertbeudeker
    @robertbeudeker
    @nuwang Thanks for the suggestion, I will look into that. Question about that. KeyCloak is not mentioned as a supported provider in Galaxy(https://galaxyproject.org/authnz/config/oidc/). How do you connect keycloak to Galaxy?
    Nuwan Goonasekera
    @nuwang
    Keycloak is the same as custos. You can specify “Custos” as the IDP provider and point to your server settings. Additionally specify the realm as in this example: https://github.com/galaxyproject/cloudlaunch-registry/blob/6c8d92ba9b9bc8b9ae11721143211b9ce931e282/app-registry.yaml#L2531
    robertbeudeker
    @robertbeudeker
    @nuwang Thank you verry much. I indeed managed to setup a working test with Keycloak an AzureAD.
    Nuwan Goonasekera
    @nuwang
    np!
    Martin Cech
    @martenson
    Hi folks, I am trying to udnerstand how container mapping works with Galaxy on k8s. Is there some sort of summary that explains this mapping?
    I am looking here: https://github.com/galaxyproject/galaxy-helm/blob/master/galaxy/files/rules/k8s_container_mapper.py#L71
    Enis Afgan
    @afgane
    There’s no summary (yet) but it’s hopefully relatively self-explaining. It’s defined here: https://github.com/galaxyproject/galaxy-helm/blob/e9167da89481c6ba3db39bb75d524a315f6dae09/galaxy/values.yaml#L390 where you just say for what tool id, which container to use. That’s the mapping itself, or are you trying to understand the logic of the method you’ve linked?
    Nuwan Goonasekera
    @nuwang
    There’s no documentation as such, this sample file is the closest we have I think: https://github.com/galaxyproject/galaxy-helm/blob/e9167da89481c6ba3db39bb75d524a315f6dae09/galaxy/values.yaml#L389
    Martin Cech
    @martenson
    Those mappings that you both linked are applied here, right? https://github.com/galaxyproject/galaxy-helm/blob/master/galaxy/files/rules/k8s_container_mapper.py#L79
    Nuwan Goonasekera
    @nuwang
    yes
    Martin Cech
    @martenson
    If there is no explicit hit in any of these k8s custom configs - the default is the container specified in tool xml requirements?
    Enis Afgan
    @afgane
    No, it will default to the container defined here: https://github.com/galaxyproject/galaxy-helm/blob/e9167da89481c6ba3db39bb75d524a315f6dae09/galaxy/values.yaml#L243
    (which is generally the Galaxy container)
    Nuwan Goonasekera
    @nuwang
    And if there’s no explicit mapping, Galaxy’s default mulled container resolution will apply
    Martin Cech
    @martenson
    So even if the tool specifies its own docker container, that will never be used in k8s?
    (unless explicitly configured through container_mapper_rules)
    Nuwan Goonasekera
    @nuwang
    container_mapper_rules only overrides. Otherwises, Galaxy’s defaults are what applies
    Enis Afgan
    @afgane
    I actually didn’t know a tool can specify its own container. Where does that happen?
    Martin Cech
    @martenson
    https://docs.galaxyproject.org/en/master/dev/schema.html#tool-requirements-container
    Enis Afgan
    @afgane
    Thanks. As Nuwan said, if the (override) mapping is not defined in the helm chart, Galaxy’s resolution should apply.
    Martin Cech
    @martenson
    So what happens if the resolution hits the container defined in the requirement?
    Is it going to riun that container in k8s?
    Enis Afgan
    @afgane
    Yes.
    Martin Cech
    @martenson
    Perfect, thank you.
    Martin Cech
    @martenson
    I have some more questions about k8s data persistence and handling. Is there something I should read before asking?
    In common deployments you either use nfs or Pulsar to stage in/out. Is Galaxy k8s different?
    Nuwan Goonasekera
    @nuwang
    It’s using NFS at the moment, but the plan is to migrate fully to Pulsar as the default (once Pulsar's remote object fetching is fully tested and ready)
    Martin Cech
    @martenson
    So in that way the Galaxy through Helm mimics the common deployment (file_path on NFS)?
    Nuwan Goonasekera
    @nuwang
    yes
    Martin Cech
    @martenson
    Alright, splendid. Thank you both.
    (And thanks for joining committers Nuwan :tada: )
    Nuwan Goonasekera
    @nuwang
    Np and glad to!
    Nate Coraor
    @natefoo
    Where's this dynamic-k8s-dispatcher rule live? https://github.com/galaxyproject/galaxy-helm/blob/32c428260cee3d4a46c4d645aaf7b171ac76213e/galaxy/values.yaml#L238
    I'm trying to figure out how to set cpu and memory limits and I assume it's done there?
    Nate Coraor
    @natefoo
    Ah https://github.com/galaxyproject/galaxy-helm/blob/32c428260cee3d4a46c4d645aaf7b171ac76213e/galaxy/files/rules/k8s_container_mapper.py
    This seems like it probably shouldn't return a dict if default_resource_set is unset: https://github.com/galaxyproject/galaxy-helm/blob/master/galaxy/files/rules/k8s_container_mapper.py#L68
    Nate Coraor
    @natefoo
    Ah destination params, not sure how I missed this: https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/job_conf.xml.sample_advanced#L1026
    Nuwan Goonasekera
    @nuwang
    @natefoo Did you run into an issue?
    Nate Coraor
    @natefoo
    I don't think there's a way to set any of these (or the walltime) for Pulsar coexecution pods
    Here's where it sets up the job template: https://github.com/galaxyproject/pulsar/blob/master/pulsar/client/client.py#L404
    Nuwan Goonasekera
    @nuwang
    Luke and I were just talking about that the other day. A lot of these options in the k8s runner will need to be ported over
    Nate Coraor
    @natefoo
    Yep