Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 18:03
    welcome[bot] commented #1307
  • 18:03
    dsblank opened #1307
  • 16:12

    choldgraf on master

    Add study participation notice … Merge pull request #1932 from j… (compare)

  • 16:12
    choldgraf closed #1932
  • 11:08
    dolfinus review_requested #1297
  • Jun 11 11:04
    sgibson91 opened #1932
  • Jun 11 11:02

    sgibson91 on sgibson91-patch-1

    Add study participation notice … (compare)

  • Jun 11 11:02
    sgibson91 opened #1046
  • Jun 11 11:01

    sgibson91 on sgibson91-patch-1

    Add study participation notice … (compare)

  • Jun 08 13:56
    choldgraf commented #1931
  • Jun 08 13:47
    welcome[bot] commented #1931
  • Jun 08 13:47
    vrubino opened #1931
  • Jun 08 06:50

    betatim on master

    Cleanup old resource requests Merge pull request #1930 from m… (compare)

  • Jun 08 06:50
    betatim closed #1930
  • Jun 08 03:18
    petersgiles commented #1306
  • Jun 08 03:17
    petersgiles closed #1306
  • Jun 08 03:05
    welcome[bot] commented #1306
  • Jun 08 03:05
    petersgiles labeled #1306
  • Jun 08 03:05
    petersgiles opened #1306
  • Jun 07 11:01
    manics opened #1930
YoungDon Choi
@DavidChoi76
I have to use requirments.txt and environment.yml to create MyBinder JupyterHub computational environment. So I'm trying to use environment.yml that includes requirments.txt such as pip -r requirements.txt. Also, I want to use automatically generated requirement.txt from Python virtual environment. In this case, there is some error that there are no python libraries. Is there a way to pass when there is no library? I found some solutions to pass it using customed python code with requirements.txt, but I couldn't find a way to use MyBinder configuration options.
Aaron Curtis
@foobarbecue
hi all, I'm running a single-node binderhub (jupyterhub helm chart on k3s, binderhub run from git with testing/local-binder-k8s-hub/binderhub_config.py ) and able to build images but after the build it fails to launch, with Cannot assign requested address . Any ideas or debugging tips for this?
Romain Primet
@rprimet
Hi all, at the last hubs meeting there was an interest in scheduling a meeting for users interested in authenticated binderhubs, in order to share use cases (including a POC deployment that @g-braeunlich is running) and shepherd a few related PRs. Would may 18th (tuesday) at 4PM UTC be a good time? (https://arewemeetingyet.com/UTC/2021-05-18/16:00/) Otherwise, what's the usual practice, post a topic and a doodle in the discourse forum?
Thanks!
(also, @sgibson91 , is there a way of retrieving the former hubs minutes? I couldn't find a link to the previous pages on the current hackmd. Thanks!)
4 replies
Dan Parsons
@danparsons_twitter
My team asked me to set up Jupyterhub for them, so I did that and it works great. Now they want Binderhub, and I'm trying to understand how it fits in. Deploying via helm chart on kubernetes. Looks like it comes with its own installation of jupyterhub too, is that true?
Dan Parsons
@danparsons_twitter
I'm trying to make auth work in binderhub but running into some issues. I'm using similar settings to a jupyterhub deploy.
config:
  BinderHub:
    auth_enabled: true
    use_registry: true
    image_prefix: dockerhubprefix/binder-prod-
    hub_url: https://binderhub.mycloud/
jupyterhub:
  cull:
    # don't cull authenticated users
    users: False
  custom:
    binderauth_enabled: true
  hub:
    config:
      GitHubOAuthenticator:
        client_id: redacted
        client_secret: redacted
        oauth_callback_url: https://binderhub.mycloud/hub/oauth_callback
        admin_users:
          - my-github-username
        allowed_users:
          - someoneelse
        scope:
          - read:user
      JupyterHub:
        authenticator_class: github
    redirectToServer: false
    services:
      binder:
        oauth_no_confirm: true
        oauth_redirect_uri: "http://binderhub.mycloud/oauth_callback"
        oauth_client_id: "binder-oauth-client-test"

  singleuser:
    # to make notebook servers aware of hub
    cmd: jupyterhub-singleuser
  #auth: {}
ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: "nginx-jupyterhub"
    nginx.ingress.kubernetes.io/proxy-body-size: 200m
  hosts:
    - binderhub.mycloud
This is my helm values file

When I visit binderhub, I'm shown a binderhub-branded 404 page. Looking at pod logs, I see this:

[I 210427 02:37:18 log:140] 302 GET / -> https://binderhub.mycloud/hub/api/oauth2/authorize?client_id=binder-oauth-client-test&redirect_uri=http%3A%2F%2Fbinderhub.mycloud%2Foauth_callback&response_type=code&state=[secret] (@10.128.5.123) 1.68ms
[W 210427 02:37:18 log:140] 404 GET /hub/api/oauth2/authorize?client_id=binder-oauth-client-test&redirect_uri=http%3A%2F%2Fbinderhub.mycloud%2Foauth_callback&response_type=code&state=[secret] (@10.128.5.123) 10.79ms

Any idea what's going on?

Or what I'm doing wrong? Any and all comments appreciated, thanks!
riturajsingh2015
@riturajsingh2015
Hey everyone
Logan Ward
@WardLT
I have been waiting for a build to start for around an hour after a previous build on the repo failed (sorry, did not cache the error message). Any thoughts on what the issue could be?
4 replies
Aaron Curtis
@foobarbecue

Hi all, I've got jupyterhub running and then I started binderhub with:

python3 -m binderhub -f testing/local-binder-k8s-hub/binderhub_config.py

but getting

[E 210503 19:16:47 builder:317] Tornado HTTP Timeout error: Failed to get image manifest for binder-2dexamples-2ddemo-2djulia-28add9:39a18c401b854f472ea72d9dedd7b11988ef289b
    Traceback (most recent call last):
      File "/home/aacurtis/cdao-dv-k8s/binderhub/binderhub/builder.py", line 313, in get
        image_manifest = await self.registry.get_image_manifest(*'/'.join(image_name.split('/')[-2:]).split(':', 1))
      File "/home/aacurtis/cdao-dv-k8s/binderhub/binderhub/registry.py", line 198, in get_image_manifest
        auth_resp = await client.fetch(auth_req)
    tornado.httpclient.HTTPClientError: HTTP 401: Unauthorized

Can someone tip me off to what I need to do to fix this? I assume jupyterhub isn't letting binderhub in?

oh actually it's the registry, isn't it? I'm using a local one so I guess I need to tell it to talk to localhost:5000
Aaron Curtis
@foobarbecue
do all these instructions, with the config.yaml and secret.yaml, still make sense? I saw something that jupyterhub removed some of these things?
Aaron Curtis
@foobarbecue
I'm getting
Error: execution error at (binderhub/charts/jupyterhub/templates/proxy/deployment.yaml:28:32): Proxy token must be a 32 byte random string generated with `openssl rand -hex 32`!
even though I did that
Aaron Curtis
@foobarbecue
sorry all, this is because I didn't -f secret.yaml and conf.yaml
ARafik
@ARafik
Hi, how I can install bezier package with the binder syntax please ?
8 replies
Nick Levandoski
@metric-chicken
How would I add files, for instance a custom .pip.conf to a jupyter image using jupyter-docker2repo?
adelin-b
@adelin-b
Hi, I ended up on binder because I was looking for a way to turn a .ipynb into a working dockerfile, binder seems to work very well but I do not find how to dowload the Dockerfile to run it on my own machine instead of binder servers.
mriduls
@mriduls:matrix.org
[m]
binder uses https://github.com/jupyterhub/repo2docker to convert repos to docker images, you can create the docker images locally using it.
Grégoire Lejay
@Antobiotics

Hi! I am trying to setup binderhub on already existing kubernetes cluster, but unfortunately I get

 User "developerssso:EKSGetTokenAuth" cannot get resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

My guess is that it is due to https://github.com/jupyterhub/binderhub/blob/c7cbf63e27b09d223035f8bdda2dab189ad826d1/helm-chart/binderhub/templates/rbac.yaml#L49
Is there a way to turn it off as I would preferrably not give such authorisation to it?
I have tried helm install binder jupyterhub/binderhub --version=0.2.0-n563.h72d30fb --namespace=data -f secret.yaml -f config.yaml --set imageCleaner.enabled=false but I get the same issue

If that's the wrong place for such question, sorry!

Erik Sundell
@consideRatio

@Antobiotics are you sure you want to workaround not having access to manipulate Cluster wide k8s resources? It will have many drawbacks, not just the first error you run into and workaround in BinderHub, but also in the JupyterHub Helm chart acting as a dependency.

I suggest you look into granting your k8s user (perhaps coupled to an AWS user somehow) that you are acting with sufficient permissions when working with helm and kubectl.

Grégoire Lejay
@Antobiotics
Thank you for coming back to me, I understand your point as this features seem to be very key for the good functioning of binderhub but I am having a lot push back from other teams working on the same k8s cluster and reliability teams for that reason and it is for them a "no-go". I was wondering if i could find an alternative solution other than creating a new cluster :/
Min RK
@minrk
@Antobiotics I believe the only features that require cluster role are the image cleaner and the user scheduler. If you disable them both, no cluster roles should be requested.
1 reply
binderhub:
  imageCleaner:
    enabled: false
  jupyterhub:
    scheduling:
      userScheduler:
        enabled: false
Of course, you will also lack those features - the disk will fill up with images eventually, and scheduling won't pack onto nodes, requiring manual intervention for efficient cluster scale-down.
Fernando Perez
@fperez

Hi folks! Does anyone know why a binder build would be replacing CPython with pypy??? In this PR for a work project we found in the build logs that the mamba update command at build time is updating cpython to pypy. That then causes problems loading scipy...

For now I'm brute-forcing a workaround by manually updating scipy itself, which seems to do the trick of reverting back to CPython, but I'm puzzled by this behavior. Any ideas would be appreciated.

Wolf Vollprecht
@wolfv
Hi @fperez this can sometimes happen with mamba ... one fix is to explicitly ask for CPython (using python=*=*_cpython)
However, we should really fix that by taking another deep look at how we handle "tracking_features" which are used by conda to de-prioritize packages. We can work on that next week
Fernando Perez
@fperez
Thanks a lot @wolfv for this info, very useful! Would be great to get this fixed in mamba proper, b/c it's a super weird bug that took me a long time to figure out and could easily leave people confused without a solution. It was only after reading the logs line-by-line super carefully that I noticed the pypy string in some of the package names and it all made sense. B/c the error we were getting didn't mention pypy at all, just not being able to import scipy due to a failure deep in a C callback API (which in retrospect makes sense would be a pypy bug :)
In any case, many thanks for this! I love mamba and I'm happy to see binder using it for its higher speed, so hopefully this kink can be ironed out.
Fernando Perez
@fperez

BTW, in the past three days I've been seeing a lot of

Found built image, launching...
Launching server...
Launch attempt 1 failed, retrying...

Sometimes it resolves eventually (long wait), sometimes it just times out altogether...

Is anyone else seeing the same? I looked at the status page but it shows all green and "no data" on the plots, so I'm not sure what's going on...

Min RK
@minrk
Our prometheus server has crashed and is trying to reconstruct its database on restart. However, the readiness probe is killing it every time because that takes more than 30 seconds.
Erik Sundell
@consideRatio
The readinessProbe is only marking it as not ready and k8s Service's won't reference network traffic to not-ready pods. The livenessProbe would need to be updated to be more tolerant and restart itself - or alternatively, a startupProbe if this is something that only happens during startup.
Min RK
@minrk
It does appear to be a startup thing.
Ah, the restarts are actually OOM-killer
reconstructing the db takes too much memory
Nick Levandoski
@metric-chicken
Is there a flag in repo2docker to do a clean build?
2 replies
Angus Hollands
@agoose77:matrix.org
[m]

I want to change the server to jupyter-server for a particular repo. I've tried using the start file

#!/usr/bin/env bash
echo "${@}"
exec jupyter lab "${@:3}"

but it seems that something is going wrong and Binder is failing to launch the built image server.

When I use r2d locally, I have to set --no-browser but otherwise it works. Any ideas?

humbletang
@humbletang:matrix.org
[m]
Hi there! I started a binder session with a notebook on my laptop and created an 8gb file that I would like to download over a faster connection, is there any way to reuse the authentication cookie from my server when I wget the large file by its link?
Anton Akhmerov
@akhmerov
Hi all! I've been examining the zenodo integration and I noticed that binder doesn't unpack zip files if a zenodo repo contains them. This is a pity because as far as I know, a zip file is the only way to upload stuff to zenodo if one wants to have folders.
  1. Do you think automatically unpacking zip files on build of zenodo uploads would be a good idea?
  2. If yes, in which repository should I open an issue about it (I imagine this could be implemented on different levels)?
Sarah Gibson
@sgibson91
the repo would be https://github.com/jupyterhub/repo2docker It's definitely worth opening an issue to discuss it :)
Anton Akhmerov
@akhmerov
Thanks!
sanjaydatasciencedojo
@sanjaydatasciencedojo
Hi everyone! We are looking for a way to run in Binderhub in a headless manner. We are planning to use it with executablebooks/thebe. Is it possible to achieve that?
Andreas Kuckartz
@akuckartz
Is there an outage? I tried to use mybinder but only got 404 errors. Also the diagrams on https://mybinder.readthedocs.io/en/latest/about/status.html do not look good: "Failed to fetch".
Nick Levandoski
@metric-chicken
Hello. Is mamba interchangeable with conda throughout the buildpack process? I notice that mamba is installed and installs conda but when the configuration files in .binder are call it calls conda.
Angus Hollands
@agoose77:matrix.org
[m]
Maybe better to ask over at #QuantStack_Lobby:gitter.im
Nick Levandoski
@metric-chicken
thanks