SwissDataScienceCenter/renku

A platform for repoducible, reusable and shareable data science

waldorivier

@waldorivier

Can we install a java virtual machine ?

4 replies

Thanks for your support and have a nice day!

Rok Roškar

@rokroskar

A bit short-notice, but we are kicking off the next round of Renku Tea Talks today! Topic: streamlit dashboards in renkulab. See details here: https://renku.discourse.group/t/renku-talks-ii-2020/175

pdurbin @pdurbin joins late

Rok Roškar

@rokroskar

@pdurbin there will be a recording somewhere at some point :)

btw, had the pleasure of seeing Mercé give a talk about Dataverse a few days back and she mentioned that some provenance-related work is on-going - would be great to sync on this at some point and see how Renku can fit in that picture!

Philip Durbin

@pdurbin

@rokroskar yes! And for my part I'd still like to see Renku as an "external tool" in Dataverse. This would mean Renku would need to take a DOI as a query parameter to its web interface.

Rok Roškar

@rokroskar

@pdurbin yeah the pieces for that are basically all there now. I think there is still an issue open for that and we can nudge it toward the top of the stack

Philip Durbin

@pdurbin

\o/

🎉

amirrezaie1415

@amirrezaie1415

Hello, I want to use the library "plotly" to interactively visualize my data. Even though I could install the library and also ipywidgets + JupyterLab extensions (according to the official documentation https://plotly.com/python/getting-started/ ), the plots were not visualized in the JupyterLab! Thanks in advance for your help. (I should add that following the provided instructions, I managed to use plotly on my local PC)

9 replies

Philip Durbin

@pdurbin

"I am going to give a look at Renku which seems to suit my case and to be a valid solution. :D" -- https://groups.google.com/g/dataverse-community/c/doWoXPsXsQ0/m/wB00tHgWAwAJ

2 replies

Philip Durbin

@pdurbin

😁

Philip Durbin

@pdurbin

I'm sure that person would be honored if you reach out.

Rok Roškar

@rokroskar

done!

Philip Durbin

@pdurbin

🎉

meschultheiss

@meschultheiss

Hello, I think I lost some work this morning : I tried to commit a file which was too large and then the jupyterlab env started to freeze. I managed to remove the large file from the commit and then push but I had conflicts to solve. Then the jupyterlab was so slow I closed the environment, hoping that renku would have autosaved my work. But after launching a new env, I was back to my work status from yesterday, and no trace of this morning work. What did I miss ? Any chance there is a copy of my notebook I cannot see ?
Also, what are the good practices to push a work with too large files ? e.g. an .html viz that was created for preview

7 replies

Waldo Rivier

@waldorivier_gitlab

Hello Renku's Team

is it possible to keep JupyterLab 's user setting between logoff / logon ?

2 replies

Waldo Rivier

@waldorivier_gitlab

Thanks for your help and have a nice day

waldorivier

@waldorivier

2 replies

Chandrasekhar Ramakrishnan

@ciyer

Hi @waldorivier. It took a little longer than had planned, but I have an example of a project that restores the JupyterLab settings. https://renkulab.io/projects/cramakri/jupyter-preferences

9 replies

meschultheiss

@meschultheiss

Hello renku team! I have an issue when building the docker image and therefore cannot start my environment. I have a warning "the docker image build failed".
I think the problem comes from the geopandas package installation. The same issue appears independently of pip or conda.

The logs are the following:

Running with gitlab-runner 13.3.1 (738bbe5a)
on renkulab-runner-2-1TB S25j9XrP
Preparing the "docker" executor
00:05
Using Docker executor with image docker:stable ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:6972c414f322dfa40324df3c503d4b217ccdec6d576e408ed10437f508f4181b for docker:stable ...
Preparing environment
00:02
Running on runner-s25XXX58-concurrent-0 via 6fabf7a874d3...
Getting source from Git repository
02:03
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/gitlab/marc-edouard.schultheiss/mobis2019_activityspace/.git/
error: RPC failed; curl 52 Empty reply from server
fatal: the remote end hung up unexpectedly
ERROR: Job failed: exit code 1

5 replies

Rok Roškar

@rokroskar

Hi all: tomorrow we will have another "renku tea-time talk" - this time by @pameladelgado on the topic of deploying renku on your own infrastructure. Details on discourse: https://renku.discourse.group/t/renku-talks-ii-2020/175/2

Melvin Wong

@mwong009

hi, i have a question regarding renku dataset add command

renku dataset add command automatically sets the name of the data to the last part of the url.

how do I add data to the dataset while also having the option to rename the file?

lets say the url to the data is: drive.switch.ch/index.php/s/abcdefg/download,
i run the command renku dataset add -eo my_dataset drive.switch.ch/index.php/s/abcdefg/download
the name of the link on the repo will be displayed as: data/my_dataset/download

how can i set the label of my data in the dataset to a specific name that i want?

do we have an option like renku dataset add --label custom_data_label my_dataset url://of_my/data?

Chandrasekhar Ramakrishnan

@ciyer

Hi @mwong009! I think the --destination/-d flag is what you want in this case. See https://renku.readthedocs.io/projects/renku-python/en/latest/commands.html#working-with-data

Melvin Wong

@mwong009

Hi @mwong009! I think the --destination/-d flag is what you want in this case. See https://renku.readthedocs.io/projects/renku-python/en/latest/commands.html#working-with-data

I think that creates a sub-directory. I've tried that and it doesnt rename the file, it only sets the location of the file

Chandrasekhar Ramakrishnan

@ciyer

Ah, now I think understand what you want.

Chandrasekhar Ramakrishnan

@ciyer

I just checked with the developers. This isn't possible at the moment (it used to be, but this feature was removed because it was exposed in a confusing way).

The way to handle this is to do a renku dataset add and then a renku mv to rename the file.

Melvin Wong

@mwong009

ok thanks

Rok Roškar

@rokroskar

@mwong009 do you use another command-line utility that downloads files from these links with the correct filename? I'm trying to find a link from switch drive that would work with wget and can't seem to do so.

Melvin Wong

@mwong009

@rokroskar so What i did was creating a link with my .csv file in Switchdrive and it gives me a drive.switch.ch/index.php/s/abcdefg/download link when i can then use the renku command renku dataset add -eo my_dataset drive.switch.ch/index.php/s/abcdefg/download to import it into the dataset

I can view the data using pandas.read_csv('data/my_dataset/download')

if you click on sharing in switch drive and copy the 'public link' into the url box, itopens up a new page that will show you a .../.../download link

Rok Roškar

@rokroskar

Thanks @mwong009, I can generate the link now - we should be able to process the metadata in the header that we get from that link to set the filename correctly, I'll open an issue for that. Thanks for pointing it out!

Ludger

@LudgerGoeminne_twitter

Hi All I'm not sure if this is the right place to post this, but when I create a basic R project in Renku, I can't add datasets in the "Datasets" tab and I get the following message: "Knowledge Graph integration must be activated to view the lineage and the datasets, but there is a problem with the knowledge graph integration for this project. To resolve this problem, you should contact the development team on Gitter or GitHub."

I followed that link and that's how I ended up here :)

Does anyone of you know how to solve this problem? I tried creating a public, private and internal basic R project, but all three give the same problem...

Rok Roškar

@rokroskar

Hi @LudgerGoeminne_twitter sorry you're having trouble - can you let me know which project this is?

Ludger

@LudgerGoeminne_twitter

I already removed it, but I made a new one to demonstrate it: it's called "testproject2"

I get the message when I go to "https://sv-renku.epfl.ch/projects/goeminne/testproject2/datasets"

Rok Roškar

@rokroskar

Aha I see - I don't have access to that instance of renku - could you contact your support? It may be that one of the services needs to be restarted if this is a persistent problem

amirrezaie1415

@amirrezaie1415

Hello, by mistake I draged and droped two times my 300 files in the data folder. And for each file, the message in the image is shown. Is it possible to apply "Overwrite" once for all files? The same is also happening when I have large number of files (each of which are around 20 MB ) and for each file a message is popping up saying that "are you sure you want to upload your file :) "
Thanks!

amirrezaie1415

@amirrezaie1415

I have also another question. I am uploading one large file (1 GB) by just dragging and dropping in the data folder. Previous times that I had tried, there was a green progress bar showing how much of the data has been uploaded, but this time there is nothing! (The word "Complete" in the image below is because of the previous small file that I uploaded successfully!)

2 replies

Rok Roškar

@rokroskar

Hi @amirrezaie1415 I'm not really sure about using the JupyterLab interface for uploading a lot of files (or large files) - we recommend that you make a dataset either with the renku CLI or in the renku UI and use that to handle adding files to the project. The former requires you to have the renku CLI installed on whatever machine you want to data to your project from. If you go through the web UI, you can just drag and drop.

Melvin Wong

@mwong009

Hello, I tried importing a project that I have from Github, and everything seems ok. I have one small problem, my prject in Github has an existing requirements.txt file, when i do a renku init --force, it somehow overwrites the requirements.txt file and deleted everything in it.

Where communities thrive

People

Repo info

Activity