Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
George Baxopoulos
@georgebax
ran*
Now I want to add the composer
Peter Amstutz
@tetron
I see
George Baxopoulos
@georgebax
But, according to its guide, I need to modify the nginx conf file. Which, as I understand, is created by salt according to the formula
Peter Amstutz
@tetron
yea. ok. so because we stopped supporting composer the salt formula doesn't include it.
but you should be able to add it
George Baxopoulos
@georgebax
Yes, I don't intend to add it using salt, I just want to follow the guide
Peter Amstutz
@tetron
yea, you need to do that in a way that doesn't conflict with salt managing the other config files. or you can just never run salt again.
George Baxopoulos
@georgebax
I am just looking for the workbench config file, which should have been at /etc/nginx/conf.d/arvados-workbench.conf according to the guide (I also can't find it doing a system wide find), so that I can edit it.
Isn't salt used just for the installation?
Peter Amstutz
@tetron
it's a configuration management system, it can be used for installation, update, rolling out configuration changes, etc
similar to puppet, ansible, etc
ok, so it looks like the nginx salt formula probably does something different from what our manually install docs say to do
George Baxopoulos
@georgebax
I understand that salt creates the configuration files (for nginx and arvados components). But where is the configuration file that it creates?
Peter Amstutz
@tetron
I'm trying to figure that out myself. the answer is probably in the nginx salt formula
Javier Bértoli
@javierbertoli
georgebax: give me a second, reading all you both discussed
George Baxopoulos
@georgebax
Ok Javier
Javier Bértoli
@javierbertoli
the salt installer uses the existing formulas for the different applications, so nginx's creates the configuration files under the default OS paths:
/etc/nginx/sites-available/*
then, enables them using links under /etc/nginx/sites-enabled/
George Baxopoulos
@georgebax
Oh, thank you very much Javier!
Javier Bértoli
@javierbertoli
which is Debian's family way of managing nginx's configuration
George Baxopoulos
@georgebax
Never figured they wouldn't have a .conf extension
Javier Bértoli
@javierbertoli
ah, right. It's not required (as any file under /etc/nginx/site-enabled/ is read in the way the nginx's formula manages the config), and I didn't realize of adding it. Perhaps it'd be good to add it, just to keep things clearer
George Baxopoulos
@georgebax
FYI, eventually composer was installed, and I added the necessary parts to the configuration. Now when I navigate to the /composer endpoint, I get a page consisting of just a Log in button, which prompts for a login and then it just gets me back to the same page with the Log in button
I was getting the samae behavior using arvbox btw
Ward Vandewege
@cure
@/all the arvados user group meeting is happening right now at https://meet.google.com/eig-fvsw-xvd
Peter Amstutz
@tetron
@pvanheus I don't know where you landed with your Arvados adventures but @javierbertoli has done a lot more work with the provisioning script for setting up Arvados instances
Peter Amstutz
@tetron
@/all the arvados user group meeting is happening right now at https://meet.google.com/eig-fvsw-xvd
pvanheus
@pvanheus
thansk @tetron - I've had to put that on hold for a while due to other responsibilities
Andy Thomson
@andy-thomason
Hi everyone. I'm looking to see if we can use Arvados on AWS to run batch jobs. The big question is do you have a backend for AWS Batch and if not, how easy would it be to implement?
Peter Amstutz
@tetron
hi @andy-thomason Arvados does support running jobs on AWS, it manages starting/stopping the EC2 instances directly instead of using AWS Batch
Andy Thomson
@andy-thomason
Thanks @tetron - I'll investigate further. I need to sell it to the team.
Peter Amstutz
@tetron
@/all Arvados 2.1.2 has been released! https://arvados.org/release-notes/2.1.2/
Lucas Di Pentima
@ldipenti
🎉 \o/
Jarett DeAngelis
@jdkruzr
@cure 👋🏻
Peter Amstutz
@tetron
hi @jdkruzr
Jarett DeAngelis
@jdkruzr
hey @tetron !
Ward Vandewege
@cure
hey @jdkruzr nice to see you here
Spyros Nakos
@nakosspy

Hi, I can't execute arvados-cwl-runner.
I installed arvados-cwl-runner with:

sudo apt install git
sudo apt install libcurl4-openssl-dev libssl-dev
sudo apt install python3-pip
sudo apt install curl
pip install -U setuptools
pip install arvados-cwl-runner

When I run arvados-cwl-runner, I get the following:
Traceback (most recent call last): File "/home/ubuntu/.local/bin/arvados-cwl-runner", line 8, in <module> from arvados_cwl import mainModuleNotFoundError: No module named 'arvados_cwl'
Any idea what is wrong here?

Peter Amstutz
@tetron
hi @nakosspy
Ward Vandewege
@cure
@nakosspy is it possible you also have python2 installed on this system, and that pip is actually python2's pip ?
I just tried the above in a docker container, but I used pip3 instead of pip, and it worked for me.
Peter Amstutz
@tetron
@nakosspy what version of ubuntu are you using?
Ward Vandewege
@cure
The alternative approach would be to install arvados-cwl-runner from our package repo if you are on a supported debian/ubuntu version, see https://doc.arvados.org/v2.1/install/packages.html. Then apt install python3-arvados-cwl-runner. Ubuntu 20.04 packages will be made available with the next stable Arvados release.
Spyros Nakos
@nakosspy
HI @tetron and @cure, thank you for your answers. No I don't have python2 installed, actually I made a typo. The command is pip3m not pip
OS is Ubuntu 18.04
Ward Vandewege
@cure
@nakosspy hmmm, I just tried in an Ubuntu 18.04 docker container as an unprivileged user, and it seems to work fine for me (I don't get an error when invoking ~/.local/bin/arvados-cwl-runner). I wonder what the difference is with your environment. Is there something special about the Python installation on your machine? I guess it would not hurt to check that the shebang line at the top of ~/.local/bin/arvados-cwl-runner is really calling python3, and not python2 or env python.
Ward Vandewege
@cure
@nakosspy is installing from apt package an option for you? If so, does that work?
Spyros Nakos
@nakosspy

Added arvados.list to /etc/apt/sources.list.d. Then sudo apt update --allow-unauthorized solved the problem in my local installation. Didn't have to install python3-arvados-cwl-runner.

The same thing doesn't work in my EC2 instance in AWS cloud.

Added arvados.list to /etc/apt/sources.list.d
sudo apt update --allow-unauthorized
Still get the same message: No module named 'arvados_cwl'
Now trying to install from apt-package:
ubuntu@ip-172-31-18-91:~$ sudo apt install python3-arvados-cwl-runner
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
python3-arvados-cwl-runner : Depends: python3.6 but it is not installable
E: Unable to correct problems, you have held broken packages.

This is obviously happening because I have installed python3.8 and not 3.6. Any idea how to solve this?

Can't install the apt package locally either. I get the same error.
Also verified that the shebang line at the top of ~/.local/bin/arvados-cwl-runner is really calling python3.