Hi, I was looking at https://github.com/reanahub/reana-ui and was wondering if external contributions are accepted at this stage of the project?
I tried setting up the project but looks like the backend can’t be accessed outside the CERN network.
Hi, the REANA project is still young but external contributions are welcome! What exactly did you have in mind for REANA UI? (P.S. Yes, the
reana.cern.ch instance is currently only available inside the CERN network. One can install REANA on premises though, several organisations already did.)
This is the recipe I'm using at the moment to work from home due to COVID-19 situation, what it does is basically bypassing the SSO auth by providing your access token on each request from the UI.
We should find a better solution eventually, but for now, this allows local development ignoring SSO.
hi
I'm trying to launch reana cluster locally via minikube
I launched minikube start --feature-gates="TTLAfterFinished=true" -p reana-test-env
then I'm walking through https://github.com/reanahub/reana-demo-helloworld
and I'm stuck on
$ export REANA_SERVER_URL=https://reana.cern.ch/
$ export REANA_ACCESS_TOKEN=XXXXXXXwhat are they for the local launch?
@alexander-myltsev Have you installed
reana-cluster or reana-dev helper tools? If yes, you can do things like eval $(reana-cluster setup-environment) which will set up the two environment variables for you. P.S. Depends on which REANA version you are trying locally, reana-cluster with 0.6.1, reana-dev with master branch. (reana-cluster has been deprecated when we switched to Helm.)
@tiborsimko I’m using
reana-dev, but it fails with the error that I don’t understand:source $(reana-dev setup-environment)
Error from server (NotFound): services "reana-test-env-traefik" not found
Environment variables could not be generated:
not enough values to unpack (expected 2, got 0)
source: not enough arguments
in the
reana folder downloaded from github, I have an error:$ reana-dev git-fetch -c ALL
[2020-07-10T10:11:15] reana-ui: git fetch upstream
Traceback (most recent call last):
File "/reana-test/reana-test-env/bin/reana-dev", line 11, in <module>
load_entry_point('reana==0.7.0.dev20200224', 'console_scripts', 'reana-dev')()
File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/reana-test/reana-test-env/lib/python3.7/site-packages/reana/cli.py", line 1174, in git_fetch
run_command(cmd, component)
File "/reana-test/reana-test-env/lib/python3.7/site-packages/reana/cli.py", line 662, in run_command
os.chdir(get_srcdir(component))
FileNotFoundError: [Errno 2] No such file or directory: '/reana-test/reana-ui’
OK, the above two
reana-dev git-* commands would necessitate to have forked and cloned all the REANA repositories... which is perhaps not what you are looking for? There are two options: (a) are you looking for a full developer-oriented setup that would allow you to modify REANA cluster sources such as workflow controller or job controller? Or (b) are you looking at having latest REANA installed locally in order to develop analysis workflows and without necessarily looking into altering REANA cluster component sources? Please let me know which of the two, we can try different recipes
running the cluster locally in a bunch of Docker containers is fine too
@alexander-myltsev Sorry, I have been busy. In your scenario, I would recommend to use Kind, it works well for local deployments and is swifter than Minikube. You can install Kind e.g.
yay -S kind-bin on Arch Linux and you are ready to go. In order to have the very latest bleeding-edge REANA cluster, you can proceed in a CI style as follows: (note that it may take up to half an hour to build everything from scratch the first time)
# create Python virtual environment
mkvirtualenv reana
# clone reana sources
cdvirtualenv && mkdir src && cd src
git clone -b kind --depth 1 https://github.com/reanahub/reana
cd reana
# install reana-dev helper script
pip install .
# clone the other necessary REANA sources
reana-dev git-clone -u anonymous -c CLUSTER -c CLIENT -c r-d-helloworld
# deploy REANA cluster and run hello world example
reana-dev run-ci --exclude-components=r-ui,r-a-vomsproxy -c r-d-helloworld
If this succeeds, you can run
eval $(reana-dev client-setup-environment) and use reana-client run -w myanalysis on your analyses, as with Minikube. (note the slight syntax change: "reana-dev client-setup-environment")
ah,
reana-dev run-ci… is creating it, I see
@tiborsimko the error:
[2020-07-10T20:38:10] reana: kind load docker-image kozea/wdb:3.2.5
Image: "kozea/wdb:3.2.5" with ID "sha256:8e49b70c16baa5f079863ae94d732a6aa0ab5f103ef72d0db050b5a0e6c3de77" not yet present on node "kind-control-plane", loading...
ERROR: failed to load image: command "docker exec --privileged -i kind-control-plane ctr --namespace=k8s.io images import -" failed with error: exit status 1
Command Output: unpacking docker.io/kozea/wdb:3.2.5 (sha256:eae1d26ddf8624a951e1444f462ef0b0a3ca078dda987257a40f9528691cb24f)...time="2020-07-10T17:39:00Z" level=info msg="apply failure, attempting cleanup" error="failed to extract layer sha256:fa0c3f992cbd10a0569ed212414b50f1c35d97521f7e4a9e55a9abcf47ca77e2: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/64/fs/usr/lib/python2.7/collections.py: no space left on device: unknown" key="extract-102132277-rjAc sha256:2d9c829ae3f7ff3e148e5c7c3a1cf378b0f90b79035e2fe9a8d78c63ccde4c89"
time="2020-07-10T17:39:00Z" level=warning msg="extraction snapshot removal failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device: unknown" key="extract-102132277-rjAc sha256:2d9c829ae3f7ff3e148e5c7c3a1cf378b0f90b79035e2fe9a8d78c63ccde4c89"
ctr: failed to extract layer sha256:fa0c3f992cbd10a0569ed212414b50f1c35d97521f7e4a9e55a9abcf47ca77e2: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/64/fs/usr/lib/python2.7/collections.py: no space left on device: unknown
[2020-07-10T20:38:10] reana: Command 'kind load docker-image kozea/wdb:3.2.5' returned non-zero exit status 1.
[2020-07-10T20:37:54] reana: Command 'reana-dev kind-load-docker-image -c reana' returned non-zero exit status 1.
[2020-07-10T20:34:05] reana: Command 'reana-dev cluster-create' returned non-zero exit status 1.there is plenty of RAM and SSD space on my machine
Another trick that I'm using is sharing /var/reana with host. I.e. you can create empty /var/reana on your host, and use:
reana-dev cluster-delete
reana-dev run-ci -m /var/reana:/var/reana --exclude-components=r-ui,r-a-vomsproxy -c r-d-helloworldIn this way Kind will host-mount /var/reana to the cluster, so there will be less demands on Docker volume disk space, and you'll be able to access /var/reana/... workspaces comfortably from within your host.
how should I delete everything that was installed? is
reana-dev cluster-delete enough?
Hi. I'm trying to set up a mini REANA cluster on CERN Openstack following this tutorial: https://reana-cluster.readthedocs.io/en/latest/gettingstarted.html
Other than a few minor things, I managed to get right down to the last command before encountering an error:
$ reana-cluster -f reana-cluster-CERN.yaml init
Error: Couldn't load repositories file (/afs/cern.ch/user/a/admorris/.helm/repository/repositories.yaml).
You might need to run `helm init` (or `helm init --client-only` if tiller is already installed)
[ERROR] Traefik initialization failed
Command '['helm', 'repo', 'add', 'stable', 'https://kubernetes-charts.storage.googleapis.com/']' returned non-zero exit status 1..
[ERROR] Command '['helm', 'repo', 'add', 'stable', 'https://kubernetes-charts.storage.googleapis.com/']' returned non-zero exit status 1.I documented what I did, and the output, as I went along:
https://codimd.web.cern.ch/7jlGFHh1Sh6vKmG7NGsPTg/
oh wait I didn't spot the start of the error message
I'll try what it suggests
I called
helm init and waited a while. Now I have (without the traceback):$ reana-cluster -f reana-cluster-CERN.yaml init
Error: This command needs 1 argument: chart name
[ERROR] Traefik initialization failed
Command '['helm', 'install', 'reana-traefik', 'stable/traefik', '--namespace', 'kube-system', '--values', '/afs/cern.ch/work/a/admorris/reana/reana/lib/python3.6/site-packages/reana_cluster/configurations/helm/traefik/minikube.yaml']' returned non-zero exit status 1..
[ERROR] Command '['helm', 'install', 'reana-traefik', 'stable/traefik', '--namespace', 'kube-system', '--values', '/afs/cern.ch/work/a/admorris/reana/reana/lib/python3.6/site-packages/reana_cluster/configurations/helm/traefik/minikube.yaml']' returned non-zero exit status 1.
@abmorris are you using latest master REANA components? reana-cluster is going to be deprecated in favor of helm deployment, you can follow instructions in our new (still WIP) docs site: https://docs.reana.io/development/deploying-locally/
ok I will try a local deployment before moving to openstack
thanks
I'm also thinking whether it's possible to use a volume + NFS
6 replies
This is where I am with the NFS provisioner: https://codimd.web.cern.ch/s/ryXsju3gw#NFS-share
I feel like I'm missing something. When I describe pods, the one called db is in a crash loop, and both the NFS provisioner and REANA server pod have warnings about pod has unbound immediate PersistentVolumeClaims
ok, I'm stuck. Could someone check that CodiMD link ^ and tell me where I've gone wrong?
I have an OpenStack volume connected to a PersistentVolume. The NFS provisioner is running and the PersistentVolumeClaim created by REANA is attached successfully.
The db pod is still in a crash loop, while the server and workflow-controller pods are stuck in ContainerCreating
6 replies
@abmorris Two meta-comments while inspecting the PVC situation: (1) If you plan to test some LHCb workflows, would it be feasible to use
reana.cern.ch instance? There is a possibility to access restricted resources via keytab if you wish, we have some ATLAS and CMS examples. We could perhaps talk in CERN's REANA Mattermost channel about your particular use case? (2) Concerning supporting EOS, it is currently necessary to have some Ceph/NFS shared storage for cluster nodes indeed. One can do stage-in/stage-out of data from/to EOS already, but this is not practical to do for all the workflow steps... We are planning to look into a possibility of running user workflows directly on the user EOS workspace; that might perhaps be of interest?
5 replies
HI all. We've deployed a test REANA instance on a k8s cluster at BNL. Ideally, we would want to run this in our openshift cluster instead. Has anyone successfully done this without admin-level privileges in openshift? I'm getting some errors about access to clusterrolebindings when using helm to install (potentially related to https://github.com/reanahub/reana/pull/291/). We would need to use openshift routes instead of Traefik ingress also. Also does anyone have documentation/an example of configuring REANA to use K5/non-CERN IDP auth, instead of local accounts?
Hi @hollowec, sorry for the late reply! It is now possible to install REANA easily into a particular namespace, so some of the things necessary for the RIVER deployment are now available out of the box. However we haven't looked systematically into the limited privileges yet... We can revive that thread though! Also, haven't tried with OpenShift yet (this could be less work) or with non-local non-CERN SSO options (this could be more work). Have tried to deploy on Google Cloud GKE, that was working well out of the box (using local accounts).
OK thanks @tiborsimko. It would be nice to integrate with our IDP if possible, and that would likely be needed before this could be adopted on a larger scale at BNL. If we are using local accounts, is there a way to disable the sign-up functionality on the web interface? I would prefer to create users manually on the reana server command-line (
flask user-create) as needed. Does user-create allow the assignment of a web-ui password? Also, is there a way for users to change their password via the web interface?
If you can provide an openshift-compatible helm chart, or associated helm values needed for use with openshift with the existing chart that would be great.
We used to have it exactly as you described in 0.6 and past releases, that is there was no sign-up on the web and it was only the admin person who could create user accounts. As of 0.7 release, users can sign-up by themselves, but please note that this action is not sufficient to launch any workflows yet; the user has to request access token via a web interface button, which will send an email to the admin, and it up to the admin to grant or refuse the user requests. Only after the admin grants token can the signed-up user start using the system. Would it be sufficient for you like this? Or would you stlil like to block sign-up? If so, please open an issue on GitHub, we can make the sign-up feature optional upon deployment!
Hi @tiborsimko , thanks yes I saw I needed to grant a user a token before they can use the system. The issue is if the web interface were to be opened to the world, someone could go through and create accounts for others, because there is no email verification needed to create the account. For instance, I could register an account for someone, say "someuser@bnl.gov", and then the real "someuser@bnl.gov" can't create an account unless I delete the original bogus user first. Also if it was fully open, external entities could fill up the user database with entries. Therefore, I think the signup should be an option: I will open a ticket in github. Thanks!
Hi there. I have been keeping an eye on this project for a while and I really like it :) I also have been following the development on Renku. I like features from both. 1) Reana that you can submit cwl workflows to a cluster 2) With Renku you can launch a interactive Rstudio/Jupyter session. In the past there have been attempts (in 2018) to integrate both systems what I found is correct. There is also a integration label on github (https://github.com/reanahub/reana/issues?q=label%3Aintegration%2Frenku+). What is the current stage/plans for this integration. And now that Reana-UI has an option to launch interactive session for Jupyter (reanahub/reana-ui#77) would this integration still be needed? Thanks for the feedback, looking forward to start playing with it.