Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Erik Sundell
@consideRatio
@Niko-la sounds like a very cool idea! If this is done id love to learn more about the solution!
Wayne's Bioinformatics Code Portal
@fomightez
@betatim, you'd probably be best to answer this, but anyone else familiar with the inner workings of repo2docker is welcome to comment as well. In an otherwise good answer on StackOverflow discussing how to set up a repo to work with Binder, the author commented, " Although it is possible to put the requirements.txt in the root folder, creating the "binder" folder reduced the building time from 2-4 minutes to 30s-1 minute." I stress this is an old answer (over a year old) and so maybe there was some truth to that at the time. But that time difference in the build duration sounded like coincidence or something else happening to me. In other words, I don't think placing the requirements.txt file in the root directory of the repo vs. having it in the binder subdirectory would make that much difference in build time. I was thinking of adding a comment to that effect but thought I'd best check here first.
Tim Head
@betatim
i wish i knew how we could get factor 4 speed ups in build times :)
no idea why putting it in the subdir made it faster
my hunch is maybe "caching" somewhere?
as in because you build it, then move the file then build again there might be stuff in the cache
so rebuilding it without moving the file would have also been faster
ps. awesome that you spot binder questions on SO and comment when you can!
I am still stunned from discovering https://en.wikipedia.org/wiki/Binder_Project :D
Philip Durbin
@pdurbin
:)
Wayne's Bioinformatics Code Portal
@fomightez
Thanks, @betatim .
Ryan Avery
@rbavery

Hello, thanks for the great tool! I have a question about a kernel dying issue. I'm trying to host a tutorial on binder, everything runs well except for the last bit where I try to plot this array:

san_pedro_sr_rgb
<xarray.DataArray (band: 3, y: 4091, x: 8251)>
[101264523 values with dtype=uint16]

I'm able to plot this no problem on an 8gb memory laptop and when I monitor the binder resources with top it looks like there is still a lot of free memory when trying to plot. but the plotting still hangs and eventually the kernel dies on this line

%matplotlib inline
san_pedro_sr_rgb.plot.imshow(robust=True)

Any suggestions? Is there an unmet matplotlib dependency I need?

And an additional detail is that I can plot this same datarray in histogram form no problem. It's just the imshow function that seems to hang and cause the kernel to die.
Wayne's Bioinformatics Code Portal
@fomightez
Hi @rbavery . Is it possible you can share a link to the repo so folks here can more easily investigate?
Ok, that link above is correct
Ryan Avery
@rbavery
I just realized, unfortunately the tutorial uses private data that you won't be able to download and test to investigate. I'll see if I can reproduce the error by plotting a dummy array of the same shape
Joe Hamman
@jhamman
@rbavery - any chance your binder session is running out of memory?
Ryan Avery
@rbavery
It looks like that's the case, I'm able to plot a smaller subset of the array. Thanks for the help!
Joe Hamman
@jhamman
You may try running your binder on binder.pangeo.io. We allocate a bit more memory there.
Tim Head
@betatim
@rbavery "kernel randomly dies" sounds like not enough memory. Unfortunately top is not useful because it doesn't understand that inside the container you don't have all the memory of the node available. @jhamman's suggestion is the way to go (or using a subset in the demo and telling people "gotta download it if you want the full monty" ... as a hook for people to be motivated to install stuff locally ;) )
Kozo Nishida
@kozo2
I would like to upgrade https://github.com/binder-examples/demo-julia/blob/master/Project.toml Julia version (to 1.3).
What would be the best thing to do it?
Wayne's Bioinformatics Code Portal
@fomightez
Fork it, change it, test it, and make a pull request, @kozo2 . It looks like Tim already added support for it at https://github.com/jupyter/repo2docker/tree/master/repo2docker/buildpacks/julia . Be aware the semantic versioning can be tricky. Last time when we were trying 1.2 @betatim tested it like so:
from repo2docker.buildpacks.julia import semver semver.find_semver_match("1.1.0", ["1.0.9", "1.0.11", "1.1.0"]) # -> 1.1.0 semver.find_semver_match("1.1.0", ["1.0.9", "1.0.11", "1.1.0", "1.1.2", "1.2.1"]) # -> 1.2.1
Sorry, it was davidanthoff that added support at https://github.com/jupyter/repo2docker/tree/master/repo2docker/buildpacks/julia .
Kozo Nishida
@kozo2
@fomightez Thank you for the information! I'll try it!
Deitel
@deitel_twitter
can someone point me to a tutorial that shows how to configure a GitHub repo to use the jupyter/scipy-notebook docker stack and include the repository's notebooks in the binder image?
Wayne's Bioinformatics Code Portal
@fomightez
@deitel_twitter Not knowing what you intend, my suggestions follow. But before that, let me say there are easier ways to 'binderize' a repo that don't involve the docker-stacks if you need the same packages. Suggestions based on what you share so far: According to here, this education-notebook repo contains everything in the jupyter/scipy-notebook stack and more.
I am not quite sure what notebooks you want included in the repository? When I launch examples from there, I just see simple default notebooks in the session launched from from here and no others. The session launched from the education-notebook repo contains no notebooks that I see. In general, there are a few options to get notebooks from other sources so you don't have to worry about keeping copies current. There are a few options to get contents from elsewhere. One is nbgitpuller. If you just want, the contents of one set one, I like using git clone in a start configuration file to pull in the latest version and launch time.
Deitel
@deitel_twitter
@fomightez Thx. I have my own notebooks that go with my textbook. I want to be able to launch a binder image that has all the libraries needed for my readers to execute the code in the notebooks. I'll check out nbgitpuller. Just thinking of ways to simplify software setups for certain aspects of our book's audience, such as the folks who don't have the local system resources required. Our books are used worldwide and not everyone has access to powerful enough equipment. Using Binder would allow them to do everything in their web browsers.
Wayne's Bioinformatics Code Portal
@fomightez
@deitel_twitter If you know the packages you need (i.e., all the ones in the scipy-notebook listing here), you can list them in requirements.txt, see here and here. If it gets more complex and you need some that pip cannot install, you can move to using an environment.yml file that allows mixing conda and pip installable dependencies. See here and examples here and here. If that isn't sufficient, you can elevate things even further. For example, often I'll also need to additional use apt.txt if it is something apt-getusually. And sometimes you'll also need add in use of postBuild and start. All those are listed here. So while you can use a Dockerfile, generally you don't need one and will have an easier time with the standard repo2docker configuration files.
Philip Durbin
@pdurbin
It feels SOOO good to have just posted this "Binder supports Dataverse DOIs for Jupyter Notebooks" message to the Dataverse mailing list: https://groups.google.com/d/msg/dataverse-community/K7vKWSPLaQY/8go3fvV3AwAJ . I encouraged everyone to install Binder. Thanks for merging @choldgraf and thanks for all the discussion on the community call the other week!
Chris Holdgraf
@choldgraf
wohoo!
Erik Sundell
@consideRatio
Wieeeeee!!!! Congratulations Philip and well done everybody involved!!
Philip Durbin
@pdurbin
Thanks! The retweets and likes are starting to come in. :) https://twitter.com/philipdurbin/status/1204472858413371392
Kirstie Whitaker
@KirstieJane
SO COOL! Way to go @pdurbin and the Binder team!
Philip Durbin
@pdurbin
Super exciting. Next, I'd like to add Binder to a future version of http://guides.dataverse.org/en/4.18.1/admin/integrations.html#analysis-and-computation but I'm not sure how best to word what Binder is. Does anyone here feel like making a pull request? We use Sphinx and reStructuredText and I'd show you which file to edit.
Kozo Nishida
@kozo2
https://github.com/yuvipanda/jupyter-desktop-server is great. However, when install.R and runtime.txt are added to the binder directory, noVNC does not start well.
Do you have any idea about this?
Of course I asked jupyter-desktop-server author yuvipanda, but let me ask the question here too.
https://twitter.com/kozo2/status/1204742785582022656
Kozo Nishida
@kozo2
I submitted an issue about this yuvipanda/jupyter-desktop-server#13
Sarah Gibson
@sgibson91
@zu_faul_twitter I'd recommend looking at this repo to be able to save your work https://github.com/manics/jupyter-offlinenotebook
Sarah Gibson
@sgibson91
I was just about to open an issue RE: lack of a search box on the new style homepage of the docs when I found @choldgraf has already opened a PR to fix it! :tada: Thank you! pandas-dev/pandas-sphinx-theme#55 Fingers crossed that it's merged soon!
Philip Durbin
@pdurbin
@betatim @choldgraf et al., comments and feedback on how I'm trying to describe Binder as a new integration from the Dataverse perspective in this pull request are welcome! IQSS/dataverse#6453
Chris Holdgraf
@choldgraf
@sgibson91 😁
Wayne's Bioinformatics Code Portal
@fomightez
Came across someone wondering why their animated plot is much slower in a MyBinder-served session vs local. Launches from https://mybinder.org/v2/gh/nathraim/test/master?filepath=rotating_line.ipynb and the repo is at https://github.com/nathraim/test/ . It uses ipywidgets and bqplot. I hope to play more with it later and see if something is obvious, but posting it here in case someone notices something in the meantime.
Angus Hollands
@agoose77
@fomightez the most obvious answer to me is that it's running over a network.
Wayne's Bioinformatics Code Portal
@fomightez
Thanks @agoose77 . That would suggest the latency being seen is not being caused on the backend and making more complex calculations shouldn't cause more noticeable deterioration as feared, it seems?
Wayne's Bioinformatics Code Portal
@fomightez
oh, wait. In the discourse posting that came after the stackoverflow post, the person states they are adding conditions and events and then it does get worse.
Daniel Sokil
@s0kil
Does Binder have options to use CPU or GPU?
Sarah Gibson
@sgibson91
mybinder.org is CPU only, but you can deploy your own BinderHub onto a GPU cluster :)