by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    grumbert
    @grumbert
    I'll bounce our confiig over to you guys for a final sanity checking
    Peter Amstutz
    @tetron
    so, we fixed the keepstore timestamp issue, @tomclegg also noted some reporting quirks from keep-balance that we'll improve in a future version
    incidentally, the plan is to have a 2.1 release of Arvados by the end of next week
    grumbert
    @grumbert
    yeah we'll look at going straight to 2.1 (if the fix is in there ofc)
    George Baxopoulos
    @georgebax
    Hello, quick question. What is the relationship between a Workflow and a Container? I understand that a workflow creates a container request upon execution, but I fail to understand how the above 2 are connected. Is the container just a docker container with the required environment to run the workflow? (If there is a class diagram or something somewhere that would be very helpful) Thanks !
    Peter Amstutz
    @tetron
    a Workflow record just stores a CWL workflow. when you hit "run" on Workbench, it creates a ContainerRequest with command: arvados-cwl-runner and the workflow description. a Container is created to fulfill a container request
    1 reply
    Evan Clark
    @djevo1_gitlab
    I saw a few tickets about this error before but I'm trying to execute a workflow and not sure what is causing this issue, I'm only getting 403/404 on the container_requests endpoint.
    Traceback (most recent call last):
      File "/usr/local/lib/python3.7/site-packages/arvados_cwl/arvcontainer.py", line 308, in run
        ).execute(num_retries=self.arvrunner.num_retries)
      File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
        return wrapped(*args, **kwargs)
      File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 840, in execute
        raise HttpError(resp, content, uri=self.uri)
    arvados.errors.ApiError: <HttpError 403 when requesting https://t4b0m.arvados.genomics.mscic.io/arvados/v1/container_requests?alt=json returned "#<ArvadosModel::PermissionDeniedError: ArvadosModel::PermissionDeniedError> (req-2w2xrhl523m5u63oq0ve)">
    Peter Amstutz
    @tetron
    @djevo1_gitlab it is a permission denied error, impossible to say why without more context
    Evan Clark
    @djevo1_gitlab
    @tetron I don't have much more than that, the nginx logs for the api server show a 403 return to the request. I'm trying to submit a workflow using arvados-cwl-runner.
    Peter Amstutz
    @tetron
    did you try the --debug flag?
    the main reason I could think of that happening is that you're submitting with a user that doesn't have write access to the parent project
    Evan Clark
    @djevo1_gitlab
    Let me check, I created the parent project and the collection and am using my own api token.
    ERROR Overall process status is permanentFail
    DEBUG EXIT run 1600955928.8620968 0.10457801818847656
    DEBUG ENTER jobiter 1600955928.8624277
    DEBUG EXIT jobiter 1600955928.8625076 7.987022399902344e-05
    DEBUG {'t4b0m-bi6l4-70f52aac21289de4': OrderedDict([('href', '/keep_services/t4b0m-bi6l4-70f52aac21289de4'), ('kind', 'arvados#keepService'), ('etag', '6huoks1p2a11eo433g946wqr2'), ('uuid', 't4b0m-bi6l4-70f52aac21289de4'), ('owner_uuid', 't4b0m-tpzed-000000000000000'), ('created_at', '2020-07-29T19:44:38.467848000Z'), ('modified_by_client_uuid', None), ('modified_by_user_uuid', 't4b0m-tpzed-000000000000000'), ('modified_at', '2020-07-29T19:44:38.467848000Z'), ('service_host', 'keep.t4b0m.arvados.genomics.mscic.io'), ('service_port', 443), ('service_ssl_flag', True), ('service_type', 'proxy'), ('read_only', False), ('_service_root', 'https://keep.t4b0m.arvados.genomics.mscic.io:443/')])}
    DEBUG 99914b932bd37a50b983c5e7c90ae93b+2: ['https://keep.t4b0m.arvados.genomics.mscic.io:443/']
    DEBUG Pool max threads is 1
    DEBUG Request: PUT https://keep.t4b0m.arvados.genomics.mscic.io:443/99914b932bd37a50b983c5e7c90ae93b
    INFO PUT 200: 2 bytes in 196.9616413116455 msec (0.000 MiB/sec)
    DEBUG KeepWriterThread <KeepWriterThread(Thread-3, started daemon 140670983468800)> succeeded 99914b932bd37a50b983c5e7c90ae93b+2 https://keep.t4b0m.arvados.genomics.mscic.io:443/
    INFO Final output collection 282fbb8a925551343e98e762fa1ec1be+57 "Output of Test.cwl (2020-09-24T13:58:49.136Z)" (t4b0m-4zz18-iempw10x5f05e4n)
    {}
    WARNING Final process status is permanentFail
    17 replies
    Michael R. Crusoe
    @mr-c
    Hey all, slurm seems to be missing from the Arvados salt recipes? https://github.com/arvados/arvados-formula/search?q=slurm
    seems to be important for crunch :-P
    Peter Amstutz
    @tetron
    that's probably right, we've only used salt for the cloud installs so far
    which don't use slurm
    Sas Swart
    @SasSwart
    @tetron We're doing openNebula/Cloudstack. Do we need slurm?
    Peter Amstutz
    @tetron
    the cloud dispatcher for compute doesn't support OpenStack at the moment. It could be added.
    Michael R. Crusoe
    @mr-c
    So, can we use a static slurm cluster instead of adding OpenStack support to the code?
    Peter Amstutz
    @tetron
    yes
    Michael R. Crusoe
    @mr-c
    Cool. Is there a public repo with slurm debian packages?
    Peter Amstutz
    @tetron
    what do you mean? slurmd is in debian
    Sas Swart
    @SasSwart
    We're looking for a munge package at the moment
    Ward Vandewege
    @cure
    also in Debian
    Michael R. Crusoe
    @mr-c
    @tetron true, I didn't know if the 18.08.5.2 version in Debian stable was too old
    Ward Vandewege
    @cure
    no, it's fine (we used to run with much older slurm versions)
    Peter Amstutz
    @tetron
    there's also a munge package in Debian
    Michael R. Crusoe
    @mr-c
    Our mistake! Their GitHub repo didn't mention the Debian package
    Peter Amstutz
    @tetron
    it is almost certainly a dependency, you can just install slurmd and slurm-client
    Michael R. Crusoe
    @mr-c
    Thanks for the clarification.
    Is there a list of requirements for adding OpenStack/OpenNebula support to Arvados? (list of APIs to implement, etc)
    Peter Amstutz
    @tetron
    what kind of support? compute support? object store support? deploy support?
    Michael R. Crusoe
    @mr-c
    Maybe https://doc.arvados.org/v2.0/install/crunch2-slurm/install-dispatch.html could be updated to mention the Debian packages first, instead of the "install SLURM from scratch" link :-)
    Compute for sure
    I think @SasSwart has instructions for deployment to OpenNebula (probably works for OpenStack as well, but that isn't tested yet)
    I filed https://dev.arvados.org/issues/16883 about updating the Aravados docs
    Michael R. Crusoe
    @mr-c
    I am still confused by one thing: I thought Arvados on AWS still used slurm?
    Peter Amstutz
    @tetron
    no
    it used to
    it doesn't any more since Arvados 2.0
    Michael R. Crusoe
    @mr-c
    So I was half right :-D I'll take it
    Which library do you use to spin up new Compute nodes on AWS?
    Peter Amstutz
    @tetron
    Michael R. Crusoe
    @mr-c
    404 ?
    Peter Amstutz
    @tetron
    yes that one
    Michael R. Crusoe
    @mr-c
    Thanks. @SasSwart has agreed to write a slurm-leader and slurm-follower roles for donation to the arvados saltstack as optional pieces. Unless one of you all already have something written?
    Peter Amstutz
    @tetron
    I don't think so