tiborsimko on 0.7.0-alpha.1
reana-team on gh-pages
Update index.yaml Signed-off-b… (compare)
reana-team on reana-0.7.0-alpha.1
tiborsimko on master
release: 0.7.0-alpha.1 (compare)
tiborsimko on master
installation: packaging depende… (compare)
tiborsimko on master
cli: release-docker * Closes #… (compare)
tiborsimko on 0.7.0a1
tiborsimko on master
installation: bump shared modul… pip-compile upgrade of dependen… release: 0.7.0a1 (compare)
tiborsimko on 0.7.0a1
reana-client, for example
reana-client -l DEBUG ping
kubect logsfor interesting pods, for example
kubectl get podsand
kubectl logs server-775d8f69f4-r6jzmwhich will display any errors and Python tracebacks
Hi, I was looking at https://github.com/reanahub/reana-ui and was wondering if external contributions are accepted at this stage of the project?
I tried setting up the project but looks like the backend can’t be accessed outside the CERN network.
reana.cern.chinstance is currently only available inside the CERN network. One can install REANA on premises though, several organisations already did.)
I'm trying to launch
reana cluster locally via
minikube start --feature-gates="TTLAfterFinished=true" -p reana-test-env
then I'm walking through https://github.com/reanahub/reana-demo-helloworld
and I'm stuck on
$ export REANA_SERVER_URL=https://reana.cern.ch/ $ export REANA_ACCESS_TOKEN=XXXXXXX
what are they for the local launch?
reana-devhelper tools? If yes, you can do things like
eval $(reana-cluster setup-environment)which will set up the two environment variables for you. P.S. Depends on which REANA version you are trying locally,
reana-clusterhas been deprecated when we switched to Helm.)
reana-dev, but it fails with the error that I don’t understand:
source $(reana-dev setup-environment) Error from server (NotFound): services "reana-test-env-traefik" not found Environment variables could not be generated: not enough values to unpack (expected 2, got 0) source: not enough arguments
reanafolder downloaded from github, I have an error:
$ reana-dev git-fetch -c ALL [2020-07-10T10:11:15] reana-ui: git fetch upstream Traceback (most recent call last): File "/reana-test/reana-test-env/bin/reana-dev", line 11, in <module> load_entry_point('reana==0.7.0.dev20200224', 'console_scripts', 'reana-dev')() File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/reana-test/reana-test-env/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/reana-test/reana-test-env/lib/python3.7/site-packages/reana/cli.py", line 1174, in git_fetch run_command(cmd, component) File "/reana-test/reana-test-env/lib/python3.7/site-packages/reana/cli.py", line 662, in run_command os.chdir(get_srcdir(component)) FileNotFoundError: [Errno 2] No such file or directory: '/reana-test/reana-ui’
reana-dev git-*commands would necessitate to have forked and cloned all the REANA repositories... which is perhaps not what you are looking for? There are two options: (a) are you looking for a full developer-oriented setup that would allow you to modify REANA cluster sources such as workflow controller or job controller? Or (b) are you looking at having latest REANA installed locally in order to develop analysis workflows and without necessarily looking into altering REANA cluster component sources? Please let me know which of the two, we can try different recipes
yay -S kind-binon Arch Linux and you are ready to go. In order to have the very latest bleeding-edge REANA cluster, you can proceed in a CI style as follows: (note that it may take up to half an hour to build everything from scratch the first time)
# create Python virtual environment mkvirtualenv reana # clone reana sources cdvirtualenv && mkdir src && cd src git clone -b kind --depth 1 https://github.com/reanahub/reana cd reana # install reana-dev helper script pip install . # clone the other necessary REANA sources reana-dev git-clone -u anonymous -c CLUSTER -c CLIENT -c r-d-helloworld # deploy REANA cluster and run hello world example reana-dev run-ci --exclude-components=r-ui,r-a-vomsproxy -c r-d-helloworld
eval $(reana-dev client-setup-environment)and use
reana-client run -w myanalysison your analyses, as with Minikube. (note the slight syntax change: "reana-dev client-setup-environment")
reana-dev run-ci…is creating it, I see
@tiborsimko the error:
[2020-07-10T20:38:10] reana: kind load docker-image kozea/wdb:3.2.5 Image: "kozea/wdb:3.2.5" with ID "sha256:8e49b70c16baa5f079863ae94d732a6aa0ab5f103ef72d0db050b5a0e6c3de77" not yet present on node "kind-control-plane", loading... ERROR: failed to load image: command "docker exec --privileged -i kind-control-plane ctr --namespace=k8s.io images import -" failed with error: exit status 1 Command Output: unpacking docker.io/kozea/wdb:3.2.5 (sha256:eae1d26ddf8624a951e1444f462ef0b0a3ca078dda987257a40f9528691cb24f)...time="2020-07-10T17:39:00Z" level=info msg="apply failure, attempting cleanup" error="failed to extract layer sha256:fa0c3f992cbd10a0569ed212414b50f1c35d97521f7e4a9e55a9abcf47ca77e2: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/64/fs/usr/lib/python2.7/collections.py: no space left on device: unknown" key="extract-102132277-rjAc sha256:2d9c829ae3f7ff3e148e5c7c3a1cf378b0f90b79035e2fe9a8d78c63ccde4c89" time="2020-07-10T17:39:00Z" level=warning msg="extraction snapshot removal failed" error="write /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db: no space left on device: unknown" key="extract-102132277-rjAc sha256:2d9c829ae3f7ff3e148e5c7c3a1cf378b0f90b79035e2fe9a8d78c63ccde4c89" ctr: failed to extract layer sha256:fa0c3f992cbd10a0569ed212414b50f1c35d97521f7e4a9e55a9abcf47ca77e2: write /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/64/fs/usr/lib/python2.7/collections.py: no space left on device: unknown [2020-07-10T20:38:10] reana: Command 'kind load docker-image kozea/wdb:3.2.5' returned non-zero exit status 1. [2020-07-10T20:37:54] reana: Command 'reana-dev kind-load-docker-image -c reana' returned non-zero exit status 1. [2020-07-10T20:34:05] reana: Command 'reana-dev cluster-create' returned non-zero exit status 1.
there is plenty of RAM and SSD space on my machine
Another trick that I'm using is sharing
/var/reana with host. I.e. you can create empty
/var/reana on your host, and use:
reana-dev cluster-delete reana-dev run-ci -m /var/reana:/var/reana --exclude-components=r-ui,r-a-vomsproxy -c r-d-helloworld
In this way Kind will host-mount
/var/reana to the cluster, so there will be less demands on Docker volume disk space, and you'll be able to access /var/reana/... workspaces comfortably from within your host.
Hi. I'm trying to set up a mini REANA cluster on CERN Openstack following this tutorial: https://reana-cluster.readthedocs.io/en/latest/gettingstarted.html
Other than a few minor things, I managed to get right down to the last command before encountering an error:
$ reana-cluster -f reana-cluster-CERN.yaml init Error: Couldn't load repositories file (/afs/cern.ch/user/a/admorris/.helm/repository/repositories.yaml). You might need to run `helm init` (or `helm init --client-only` if tiller is already installed) [ERROR] Traefik initialization failed Command '['helm', 'repo', 'add', 'stable', 'https://kubernetes-charts.storage.googleapis.com/']' returned non-zero exit status 1.. [ERROR] Command '['helm', 'repo', 'add', 'stable', 'https://kubernetes-charts.storage.googleapis.com/']' returned non-zero exit status 1.
I documented what I did, and the output, as I went along:
helm initand waited a while. Now I have (without the traceback):
$ reana-cluster -f reana-cluster-CERN.yaml init Error: This command needs 1 argument: chart name [ERROR] Traefik initialization failed Command '['helm', 'install', 'reana-traefik', 'stable/traefik', '--namespace', 'kube-system', '--values', '/afs/cern.ch/work/a/admorris/reana/reana/lib/python3.6/site-packages/reana_cluster/configurations/helm/traefik/minikube.yaml']' returned non-zero exit status 1.. [ERROR] Command '['helm', 'install', 'reana-traefik', 'stable/traefik', '--namespace', 'kube-system', '--values', '/afs/cern.ch/work/a/admorris/reana/reana/lib/python3.6/site-packages/reana_cluster/configurations/helm/traefik/minikube.yaml']' returned non-zero exit status 1.
This is where I am with the NFS provisioner: https://codimd.web.cern.ch/s/ryXsju3gw#NFS-share
I feel like I'm missing something. When I
describe pods, the one called
db is in a crash loop, and both the NFS provisioner and REANA
server pod have warnings about
pod has unbound immediate PersistentVolumeClaims
ok, I'm stuck. Could someone check that CodiMD link ^ and tell me where I've gone wrong?
I have an OpenStack volume connected to a
PersistentVolume. The NFS provisioner is running and the
PersistentVolumeClaim created by REANA is attached successfully.
db pod is still in a crash loop, while the
workflow-controller pods are stuck in
reana.cern.chinstance? There is a possibility to access restricted resources via keytab if you wish, we have some ATLAS and CMS examples. We could perhaps talk in CERN's REANA Mattermost channel about your particular use case? (2) Concerning supporting EOS, it is currently necessary to have some Ceph/NFS shared storage for cluster nodes indeed. One can do stage-in/stage-out of data from/to EOS already, but this is not practical to do for all the workflow steps... We are planning to look into a possibility of running user workflows directly on the user EOS workspace; that might perhaps be of interest?