Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
pvanheus
@pvanheus
so I removed 'format' and now got 'Memory cgroup out of memory: Kill process 31056 (metaeuk) score 1007 or sacrifice child' - this is a rather high memory step but I'd expect this machine to have enough RAM to run it... does arvados impose strict default memory limits when no memory limit is otherwise specified?
Michael R. Crusoe
@mr-c
@pvanheus I think there's a default memory allowance, yes
@pvanheus remind me where you have installed Arvados?
pvanheus
@pvanheus
on a VM with 16 GB RAM, 4 vCPUs... Ubuntu 18.04
Michael R. Crusoe
@mr-c
👍
pvanheus
@pvanheus
if I run the same workflow with cwltool on the same machine it runs to completion...
Peter Amstutz
@tetron
@pvanheus Yes Arvados sets a strict upper memory limit by default you'll need to add a ResourceRequirement
pvanheus
@pvanheus
ok. I added a ramMin of 4096 and it is still getting canceled. Going to try bump it further
pvanheus
@pvanheus
hm, seems the workflow as a whole gets 1GB of RAM despite the metaeuk step requesting more
Joshua C. Randall
@jrandall
I came late to the community meeting - guess I missed it?
Michael R. Crusoe
@mr-c
@jrandall I think it was canceled given the USA holiday yesterday :-)
Joshua C. Randall
@jrandall
Ah, that makes sense!
Peter Amstutz
@tetron
Sorry about that I should have posted something about it being canceled
Peter Amstutz
@tetron
@pvanheus did you get it working? I think you just need to make sure you're requesting the right amount of RAM in the CWL. cwltool has a --strict-memory-limit flag which is more like the Arvados behavior
also crunch-dispatch-local doesn't do any resource management, so it can overburden the node if you are running a bunch of very large tasks in parallel, if you expect to be using it for production jobs we'll need to add something like what cwltool parallel mode does
pvanheus
@pvanheus
@tetron nope... I added a ResourceRequirement to the workflow as a whole but when I ran it the workflow still had the same memory limits specified (1G RAM essentially). I'll post more details to the forum...
Peter Amstutz
@tetron
maybe the tool provides is own ResourceRequirement that takes precedence?
Ibrahim Cagri Kurt
@ibrahimkurt
Could DNS configuration part of Salt installation (single host) redundant? Looks like my /etc/hosts is already configured with a more comprehensive list of modules. Just a feedback.

Trying to get arv-copy to work on a fresh Salt installation to get the tutorial data from pirca. Running into an error of:

ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)

I believe all 3 .config files are set properly (pirca.config, arva2.config, settings.config). arv current user is functional. Any ideas?

Ibrahim Cagri Kurt
@ibrahimkurt
sorry just realizing the config.yml edits that needs to be introduced. am I supposed to provide SystemRootToken, ManagementToken etc like in a manual installation?
Peter Amstutz
@tetron
the SSL error means you need to set ARVADOS_API_HOST_INSECURE=1
or you need to install the self-signed certificate that the salt install generated
Peter Amstutz
@tetron
@javierbertoli for the self-install salt scripts, either the the user really needs to install the self-signed cert in the browser's root authority, because there's a various situations where the browser (Firefox at least) never asks you if want to accept the certificate, it just silently fails
Javier BĂ©rtoli
@javierbertoli
cute :P
Peter Amstutz
@tetron
and then it's better to create a private root and then issue server certificates signed by that root rather than using the root directly, because then you can change the server certificates without having to load a new root certificate everywhere
this is how I solved this problem in arvbox (and was suggesting you do the same thing in the salt formula)
I'm also not entirely sure if the browser even lets you use a root authority as a server certificate
Ibrahim Cagri Kurt
@ibrahimkurt
It was ARVADOS_API_HOST_INSECURE=true now tried ARVADOS_API_HOST_INSECURE=1. I guess in shell 0 is true? I am getting the same error. Could you please point me towards the certificate salt generated?
Peter Amstutz
@tetron
@javierbertoli ^^ where's the certificate generated?
Javier BĂ©rtoli
@javierbertoli
https://github.com/saltstack-formulas/arvados-formula/blob/master/test/salt/states/example_add_snakeoil_certs/init.sls is the script that generates the certs in the formula, the scripts are stored under debian's standard paths, /etc/ssl/certs/ and /etc/ssl/private/
/etc/ssl/private/ssl-cert-snakeoil.key and /etc/ssl/certs/ssl-cert-snakeoil.pem
Peter Amstutz
@tetron
do you need to run update-ca-certificates ?
Ibrahim Cagri Kurt
@ibrahimkurt
seems like # update-ca-certificates did not help
Javier BĂ©rtoli
@javierbertoli
not sure that help. There's no CA here, it's a self-signed cert.
Peter Amstutz
@tetron
so then we should be creating a private CA then
Javier BĂ©rtoli
@javierbertoli
yep.
sad but true :D
I'll see to fix this
Ibrahim Cagri Kurt
@ibrahimkurt
thanks!
in theory is it possible to install git & composer services manually on top of the salt installation by modifying the config.yml etc?
Peter Amstutz
@tetron
yes
grumbert
@grumbert
quick question. trying to use keepstore (v2.0.4) with S3 but I'm getting the following error : "error":"error initializing volume xxxx-nyw5e-0000000000000000: unrecognized region \"eu-west-2\"; try specifying endpoint instead"
Peter Amstutz
@tetron
that's weird
grumbert
@grumbert
DriverParameters: Bucket: xxxx AccessKey: xxxx SecretKey: xxxx Region: eu-west-2 Endpoint: "" LocationConstraint: false IndexPageSize: 1000 ConnectTimeout: 0s ReadTimeout: 0s RaceWindow: 24h
that's from my config.yml
(obviously on individual lines)
eu-west-2 is a valid region as far as I can tell
grumbert
@grumbert
does it try and do a dns lookup?