Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 05 2018 15:47
    choldgraf commented #103
  • Dec 31 2017 23:58
    ablekh commented #103
  • Dec 27 2017 20:50
    choldgraf commented #103
  • Dec 23 2017 12:01
    ErenKaganNamal commented #103
  • Dec 22 2017 00:48
    rl1m opened #103
  • Dec 05 2017 11:04
    soerendip commented #34
  • Oct 27 2017 18:04
    Carreau opened #102
  • Sep 11 2017 22:39
    jakirkham commented #101
  • Sep 11 2017 21:47
    agahkarakuzu commented #101
  • Sep 11 2017 21:47
    agahkarakuzu commented #101
  • Sep 11 2017 21:41
    agahkarakuzu commented #101
  • Sep 11 2017 21:39
    agahkarakuzu commented #101
  • Sep 11 2017 21:29
    jakirkham commented #101
  • Sep 11 2017 21:00
    agahkarakuzu opened #101
  • Aug 18 2017 02:45

    choldgraf on master

    Update README.md (compare)

  • Aug 16 2017 14:21
    minrk commented #100
  • Aug 12 2017 20:52
    yuvipanda commented #100
  • Aug 12 2017 20:03
    zbeekman commented #100
  • Aug 12 2017 20:00
    yuvipanda commented #100
  • Aug 12 2017 19:58
    zbeekman commented #100
Juan-Pablo Velez
@jpvelez
where it uses a separate cluster
Min RK
@minrk
JupyterHub has the notion of a spawn form, where users can provide input to the Spawner.
Juan-Pablo Velez
@jpvelez
that uses n1-highmem-16 machine and guarantees all 104GB of ram
to that user
Min RK
@minrk
Some compute cluster deployments use this to request resources - e.g. number of nodes, GPUs, queue, etc.
If requesting different resources is likely to be common, that is probably a good place to do it.
Juan-Pablo Velez
@jpvelez
nice
makes sense
@minrk how do people usually do git integration
there's open > terminal
but some of my users aren't CLI savvy
was thinking of using ungit
Min RK
@minrk
It varies
Juan-Pablo Velez
@jpvelez
and integrating via navbar
like datalab does
Min RK
@minrk
Some folks are doing automatic commit and push at various points in time (e.g. on server shutdown)
I don't have a good model for doing git commands on a user's behalf, since I know it would never do what I wanted it to. Of course, I'm not the target audience for auto-git tools.
Adding a git UI to the notebook via an extension is certainly doable. I don't know of a particularly good existing one, though.
Juan-Pablo Velez
@jpvelez
ungit is pretty user friendly
haven't used it much tho
@minrk thanks for your time and work
i'm working on a local solution for dockerizing our notebooks / data science env first
and will be doing cloud part in a couple of months
fwiw the concept for local is
Tony Hirst
@psychemedia
@minrk can binderhub/jupyterhub be used to build/launch interlinked docker-composed containers? So eg a user could launch Jupyter notebook container + a persistent storage volume, or Jupyter notebook + PostgreSQL db (+ persistent db storage volume)?
Juan-Pablo Velez
@jpvelez
git clone <data-science-repo-made-with-cookiecutter>; cd <repo>; sb jupyter notebook -> does docker build . and docker run in the background, mounting the working and home directories into the container. so data scientists can use a fat base image and you can ensure reproducibility via docker without teaching everyone how to use docker
Min RK
@minrk
cool! I think a few projects are coming to similar conclusions
@psychemedia Yes, in theory, but I don't think any Spawners do this already.
Juan-Pablo Velez
@jpvelez
turns out to be pretty tricky O_O
will open source when i get there
Min RK
@minrk
Great!

@psychemedia

A Spawner does:

  1. <opaque, arbitrary magic>

  2. return (ip, port) of a running notebook server
Typically, that step 1. involves launching a docker container or kubernetes pod. This is what people have implemented thus far.
There's no reason, though, that it couldn't start a collection of containers, instead of a single one. That's just a Spawner that hasn't been written yet.
In Kubernetes, this means (to my not-yet-deep understanding of kube) spawning a Service instead of a single Pod.
Tony Hirst
@psychemedia
@minrk ah, ok - so spawner could run docker-compose script and use notebook as the UI component into the linked apps?
Min RK
@minrk
Precisely
A docker-compose spawner would be pretty cool
Tony Hirst
@psychemedia
Does the binderhub/jupyterhub setup require a Jupyter client? OR can it be used to manage launch and user redirect to any service that has an http/html UI? eg an RStudio or OpenRefine container? (Apols if these are really naive questions.)
Min RK
@minrk
It's very close to working with any web server, but not quite.
There's a tool used at Berkeley to add RStudio to the notebook environment
I'd like to be able to make that first-class & official
Which should become easier after the OAuth stuff gets released in 0.8.
Juan-Pablo Velez
@jpvelez
yeah that would be awesome
authenticate and pick your server flavor
Tony Hirst
@psychemedia
@minrk great - thanks; I'll keep on trying to keep up!:-)
Chris Holdgraf
@choldgraf
@jpvelez just another follow up to your and @minrk 's conversation above - whenever Jupyterlab gets a 1.0 release, it will almost certainly quickly be available within Binder / JupyterHub for users. That opens up new avenues for doing more user-friendly and/or UI-based stuff
Yuvi Panda
@yuvipanda
@jpvelez btw, jupytere/repo2docker sounds very similar to the local docker solution you are solving :) we support running it locally to almost the same workflow you mentioned
Mark Mathis
@aliasmrchips
If I add ADD jupyter_notebook_config.py /home/jovyan/.jupyter/ to my Dockerfile, with c.NotebookApp.iopub_data_rate_limit = 1000000 in the file, that should fix the IOpub data limit problem?
Juan-Pablo Velez
@jpvelez
@yuvipanda interesting! as far as i can tell what i'm working on goes a bit beyond this. the idea is to execute arbitrary commands, including but not limited to jupyter notebook, transparently inside of a docker image without knowing anything about docker: sb bq ls, sb ipython, etc