arvados/community

Q&A, support and general discussion about the Arvados project, for development see https://gitter.im/arvados/development

Peter Amstutz

@tetron

actually what you probably want is the "contents" end point

api.groups().contents

Brad Chapman

@chapmanb:matrix.org

[m]

Ah thanks so much. So I really shouldn't be thinking of it as the root of the filesystem and just be searching for a well-known name underneath it as the root folder. I was mostly trying to using folder hierarchy to avoid searching on a name and then having someone else create something with the same name that messes up that logic.

Is there any way to create this kind of top level unambiguous name I can search on in a way that is not likely to break?

Brad Chapman

@chapmanb:matrix.org

[m]

I decided to handle this by adding a property on the top level folder that we could filter the search by. This seems to work pretty cleanly and prevents any obvious ways of breaking the logic.

Peter Amstutz

@tetron:matrix.org

[m]

yes, that's a good way to do it

in general, everything is owned by another user, or a project

stuff can also be owned by the system user

Adam Cowan

@adamcowan

Hi all - I am having trouble deploying a single host / single hostname configuration using the provision.sh script. I have followed the instructions here and copied the local.params.example.single_host_single_hostname to the local.params file, and the config_examples/single_host/single_hostname directory to local_config_dir, however I am running into the salt error below. Any help would be greatly appreciated!

Writing pillars and states
sed: can't read /srv/pillars/nginx_download_configuration.sls: No such file or directory
sed: can't read /srv/pillars/nginx_collections_configuration.sls: No such file or directory
[ERROR   ] Specified SLS 'aws_credentials' in environment 'base' is not available on the salt master
[ERROR   ] Template was specified incorrectly: False
[ERROR   ] Specified SLS 'letsencrypt' in environment 'base' is not available on the salt master
[ERROR   ] Template was specified incorrectly: False
[CRITICAL] Pillar render error: Specified SLS 'aws_credentials' in environment 'base' is not available on the salt master
[CRITICAL] Pillar render error: Specified SLS 'letsencrypt' in environment 'base' is not available on the salt master
[INFO    ] Loading fresh modules for state activity
local:
    Data failed to compile:
----------
    Pillar failed to render with the following messages:
----------
    Specified SLS 'aws_credentials' in environment 'base' is not available on the salt master
----------
    Specified SLS 'letsencrypt' in environment 'base' is not available on the salt master
Removing .psql file

Javier Bértoli

@javierbertoli:matrix.org

[m]

@adamcowan: can you give me the commit for the provision script you're using, and some more details (are you running it on aws, locally, are you using your own certs?). DM me, if you want, I'll check and try see what's going on as soon as possible

Michael Crusoe

@mr-c:matrix.org

[m]

Kueue: Kubernetes-native Job Queueing proposal
https://bit.ly/kueue-apis

Ward Vandewege

@cure

@adamcowan I can reproduce the problem, I made a ticket (https://dev.arvados.org/issues/18785), will fix asap and let you know.

1 reply

Ward Vandewege

@cure

@adamcowan apologies for the delay, we have pushed fixes for the bug you encountered. We have also revamped the documentation, cf. https://doc.arvados.org/v2.3/install/salt-single-host.html. Can you try again? You should start over with a fresh copy of local.params and local_config_dir, from the tip of the 2.3-release branch

Peter Amstutz

@tetron

@/all Hi everyone, Arvados 2.3.3 has been release! https://arvados.org/release-notes/2.3.3/

Michael R. Crusoe

@mr-c:matrix.org

[m]

Nice. Is there a CWL conformance test for https://dev.arvados.org/issues/18723 ?

Peter Amstutz

@tetron

ah, no, there could be. there's a cwltool test for it. add it to the to-do list?

Michael R. Crusoe

@mr-c:matrix.org

[m]

Added to common-workflow-language/cwl-v1.2#54

Ward Vandewege

@cure

@adamcowan were you able to get Arvados installed?

Peter Amstutz

@tetron

@/all Arvados 2.4.0 has been released! New in this version: GPU support, better spot instance support, faster arv-mount, new workbench2 UI for managing users & groups, check it out! https://arvados.org/release-notes/2.4.0.html

Peter Amstutz

@tetron

@/all the biweekly Arvados user group meeting is happening soon, details here: https://forum.arvados.org/t/arvados-user-group-video-chat/47/13?u=tetron

Gudule JR

@GuduleJR_twitter

Dear all. Just to thank's all of you to this great version of Arvados. I could install the API and controller with provate certificate, without any issue, using teh rpovision script. Some modifications needed to install some keepstore, and could do a arv-put. That's a great improvement from the manual install. I'll follow with the workbench in this day... but i would thank's you a lot for this!!!

Ward Vandewege

@cure

@GuduleJR_twitter that's great to hear, thank you for the feedback! Feel free to let us know if there are further things we could improve/simplify.

wjt

@erow

Hello! I got an error when installing composer:

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! electron-chromedriver@1.8.0 install: node ./download-chromedriver.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the electron-chromedriver@1.8.0 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Peter Amstutz

@tetron

@erow hi there, unfortunately we don't support Composer any more, where did you find it?

wjt

@erow

There is a link below Home project.

I also have a problem on runing the process which is quened but not running.

Peter Amstutz

@tetron:matrix.org

[m]

how did you install Arvados?

wjt

@erow

run arvbox start localdemo in a docker

Peter Amstutz

@tetron

I see

so I think there are a couple of issues

wjt

@erow

Actually, I fixed a bug that pulls an inexistent image (dev.dev20220404200812)

Peter Amstutz

@tetron

arvbox is also a developer tool so it has some references to Composer, but we actually stopped supporting it due to lack of resources, so that should just be removed

wjt

@erow

maybe the network does work well. It is common problem in China.

How to check the crunch?

Peter Amstutz

@tetron

and arvbox runs jobs using docker-in-docker but on newer Linux distributions that stopped working due to some cgroups2 issue, so it should probably default to using singularity

Peter Amstutz

@tetron:matrix.org

[m]

you can try "arvbox log crunch-dispatch-local"

if you see some error about cgroups2, it is that

wjt

@erow

Yeah. I also need to install the cloud dispatcher. It's a little hard to do it from the document.

2022-04-28_13:27:20.99879 + flock /var/lib/gopath/gopath.lock go install git.arvados.org/arvados.git/services/crunch-dispatch-local
2022-04-28_13:27:21.02748 go: downloading github.com/Azure/go-autorest/autorest v0.11.22
2022-04-28_13:27:21.02781 go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.9
2022-04-28_13:27:21.02856 go: downloading github.com/Azure/go-autorest/autorest/to v0.4.0
2022-04-28_13:27:21.02909 go: downloading github.com/jmcvetta/randutil v0.0.0-20150817122601-2bb1b664bcff
2022-04-28_13:27:21.03126 go: downloading golang.org/x/crypto v0.0.0-20211117183948-ae814b36b871
2022-04-28_13:27:21.03668 go: downloading github.com/Azure/azure-sdk-for-go v45.1.0+incompatible
2022-04-28_13:27:21.03753 go: downloading github.com/julienschmidt/httprouter v1.2.0
2022-04-28_13:27:21.03927 go: downloading github.com/aws/aws-sdk-go v1.25.30
2022-04-28_13:27:21.05389 go: downloading github.com/creack/pty v1.1.7
2022-04-28_13:27:21.05419 go: downloading github.com/docker/docker v17.12.0-ce-rc1.0.20210128214336-420b1d36250f+incompatible
2022-04-28_13:27:21.05533 go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
2022-04-28_13:27:21.05792 go: downloading gopkg.in/src-d/go-billy.v4 v4.0.1
2022-04-28_13:27:21.05916 go: downloading gopkg.in/src-d/go-git.v4 v4.0.0

it is downloading

Peter Amstutz

@tetron

arvbox is really just a demo/test environment and not intended for real use. if you want to set up a real installation on the cloud you should use the installer

wjt

@erow

OK. I will have a try.

Thank you!

Peter Amstutz

@tetron:matrix.org

[m]

you're welcome, feel free to ask questions here or on the forum (https://forum.arvados.org/)

wjt

@erow

I found the following error: No matching sls found for 'locale' in env 'base'. I want to run all the services except computing on a host node. So I set the IPs the same following 'local.params.example.multiple_hosts' and SSL_MODE="self-signed".

tetron

@tetron:matrix.org

[m]

do you want to post a more extensive error log and description of what you tried to do on the forum?

wjt

@erow

OK. I post it in https://forum.arvados.org/t/salt-install-fails/109

Gudule JR

@GuduleJR_twitter

Really a great job for the 2.4 version. I need to make some change in the salts pillars to eliminate the "CLUSTER" part in all of the servers names : in my site, o could not add this name in the DNS. And that's run, like a charm. I really thank's you about all of this... Right now, the beginning of the most important: the administration of this site.... :-)

1 reply

Ward Vandewege

@cure

@GuduleJR_twitter glad to hear it! The salt installer does make a number of assumptions, indeed. Glad to hear you were able to make the necessary adaptations for your setup.

Where communities thrive

People

Repo info

Activity