by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Sarah Gibson
@sgibson91
I can't take all the credit, we're a team :)
InSon Zeng
@son520804

When I typed helm install jupyterhub/binderhub --version=0.2.0-n200.h305d0e8 --name=binderhub --namespace=binderhub -f secret.yaml -f config.yaml in shell, the error message is
Error: release binderhub failed: clusterroles.rbac.authorization.k8s.io "binderhub-image-cleaner" already exists

What code should I type to remove the old release and make a new one??

I can't take all the credit, we're a team :)

I genuinely appreciate your motivation to be a wonderful team player.

Sarah Gibson
@sgibson91
helm delete binderhub --purge then kubectl delete namespace binderhub
InSon Zeng
@son520804
@sgibson91 Absolutely wonderful! Then when I type kubectl --namespace=binderhub get svc proxy-public, why did I still get pending as External-IP? Should I wait until the external IP come out then I continue to work on upgrading the helm chart?
Sarah Gibson
@sgibson91
Yes :) It takes a while for all of the services to be up and running and, by default, helm doesn't leave your terminal hanging while these steps complete. So keep running kubectl --namespace=binderhub get svc proxy-public until it's no longer pending and then continue :)
You can all run kubectl -n binderhub get pods to see the status of all the pods. Once they are all Running, then your External IP should also be available
InSon Zeng
@son520804
Thank you! It's really the clearest explanation I have ever seen for Binderhub :)
How long should an average user wait until all the pods are Running? So far only binderhub-dind-79tt7 and binderhub-image-cleaner-lgvhw are ready, and other 5 pods are all pending.
Sarah Gibson
@sgibson91
It can take some time, you can try kubectl -n binderhub describe pod <name-of-pod-not-yet-running> and see if it tells you anything informative
InSon Zeng
@son520804

@sgibson91 Thank you!! I ran kubectl -n binderhub get pods again, there was no luck. Only two pods are running still. When I ran kubectl -n binderhub describe pod binder-db87658b5-rbcng, where binder-db87658b5-rbcng is a pending pod, the message shows

1) Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
2) Reason: FailScheduling
3) Message: 0/1 nodes are available: 1 Insufficient memory
In fact, I checked all 5 pending pods, all of which showed insufficient memory and fail to schedule.
Judging from these, I am worried that those pods will basically continue to be pending. Could you tell me why wasn't there sufficient memory to start a binderhub?

Or, is there any way to force the pods to run?
Sarah Gibson
@sgibson91
Maybe the machines in your cluster aren't big enough? What VMs did you deploy when you created your cluster?
InSon Zeng
@son520804
@sgibson91 Maybe. That is a good question. I followed the tutorial and did not pay much attention to what VMs I used to create my cluster. How to check this information Sarah? Thank you!
InSon Zeng
@son520804
To my best knowledge, I registered Google Cloud Kubernetes, connected Kubernetes to Docker, and not sure whether one of these serve as the VM
Sarah Gibson
@sgibson91
Run kubectl get nodes to see how many VMs are in your Kubernetes cluster and kubectl describe node <node-name> should tell you compute info such as CPU and RAM
On Azure, the minimum VM size I deploy is 8 GB RAM and 2 (v)CPUs, 3 of those in the cluster. This is usually enough to install and run BinderHub.
InSon Zeng
@son520804

Thank you! Running kubectl get nodes I got this. Looking at the output, it means that my only node is docker-desktop, which is true. (Does it mean there is 1 VM in my Kubernetes cluster?)

NAME STATUS ROLES AGE VERSION
docker-desktop Ready master 15d v1.16.6-beta.0

Sarah Gibson
@sgibson91
Oh, that's strange
I'm not really familiar with deploying on GKE but I would have expected some virtual machines in there
3 replies
InSon Zeng
@son520804

Well, maybe that's why I had no luck in getting all the pods running yesterday?
Now, looking at the output of kubectl describe node docker-desktop, there are snapshot of the messages.

Capacity:
cpu: 4
ephemeral-storage: 61255492Ki
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 2038904Ki
pods: 110
Allocatable:
cpu: 4
ephemeral-storage: 56453061334
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 1936504Ki
pods: 110

It seems to me that there are 4 CPUs, and all of those are allocatable (what does this mean?). I feel unclear about what describes the capacity of RAM and memory, respectively. Could you take a look of this?

I wonder if the allocated resource table tells me the CPU and RAM (memory). If true, then my GKE allows 2 GB RAM only, and 5 GB CPU storage? It's terrifying to me not noticing that the resources are not enough to build a Binderhub.

Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits


cpu 1300m (32%) 0 (0%)
memory 1676Mi (88%) 340Mi (17%)
ephemeral-storage 0 (0%) 0 (0%)

Sarah Gibson
@sgibson91
You need to follow these instructions to set up a kubernetes cluster: https://zero-to-jupyterhub-k8s.readthedocs.io/google/step-zero-gcp.html
Then deploy the BinderHub again
InSon Zeng
@son520804
Wow! That's interesting. Let me try and get back to you soon! Thank you!
InSon Zeng
@son520804

You need to follow these instructions to set up a kubernetes cluster: https://zero-to-jupyterhub-k8s.readthedocs.io/google/step-zero-gcp.html

Thank you for the tutorial! I finally set up the Kubernetes cluster :)
The message is now:
NAME STATUS ROLES AGE VERSION
gke-binderhub-default-pool-0c4bb788-ntmb Ready <none> 6m30s v1.16.13-gke.1
gke-binderhub-default-pool-0c4bb788-nwrj Ready <none> 6m30s v1.16.13-gke.1
gke-binderhub-default-pool-0c4bb788-w779 Ready <none> 6m30s v1.16.13-gke.1

The tutorial is absolutely helpful for starters for Binderhub.

Sarah Gibson
@sgibson91
Fab, now try installing BinderHub onto the new cluster
Clarke Iakovakis
@ciakovx
Hi there, I'll be running a workshop this week with about 35 people using Jupyter Notebooks in Binder to send API calls. You mention in the docs https://mybinder.readthedocs.io/en/latest/user-guidelines.html that a binder repo can get banned for "A large, unexpected spike in traffic that persists over time" or "Sessions that attempt to perform a lot of outgoing networking traffic." I am wondering if I need to take any steps before the workshop starts to help make sure that the Binder does not get banned, could you please help me with that? Please let me know if I need to contact someone off of Gitter. Thank you!
Sarah Gibson
@sgibson91
Hey @ciakovx, the default cap for repositories running on Binder is 100 sessions, so 35 is not going to be a large spike. Banning is also a manual process (a human specifically makes a PR adding the banned repo to a list), so now you've told us it's no longer unexpected :) I'd recommend sharing the repo so someone from the team can double check that everything's OK
InSon Zeng
@son520804

@sgibson91 Thank you for your patience! I got another question: When I want to install Jupyterhub using helm upgrade --install $RELEASE jupyterhub/jupyterhub \ --namespace $NAMESPACE \ --version 0.7.0 \ --values config.yaml
, the result turned out to be surprising: Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "apps/v1beta2"

Question: How to fix? Also, why was there no matches for deployment?

Sarah Gibson
@sgibson91
You need both the config.yaml and secret.yaml in the helm upgrade command
-f config.yaml -f secret.yaml instead of --values...
InSon Zeng
@son520804
Make sense, I hope it would work but without luck. The error message returned to me "Error: validation failed: unable to recognize "": no matches for kind "Deployment" in version "apps/v1beta2""
Sarah Gibson
@sgibson91
What version of kubectl are you runming
Kubectl version
InSon Zeng
@son520804

I think my kubectl version is v1.16.6

Messgage:
Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:26:26Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:18:29Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}

InSon Zeng
@son520804
@sgibson91 I wonder whether I have connected to a cluster to deploy binderhub actually. With no luck, I ran kubectl get pod --namespace=binderhub again, and still only the same 2 pods are running, the remaining 5 pods are still pending.
Sarah Gibson
@sgibson91
Yeah, sorry, we've reached the edge of what I know/can do over chat
InSon Zeng
@son520804
Thank you so much Sarah! I can't have learnt that much without your input.
Clarke Iakovakis
@ciakovx
Thanks for the fast answer, @sgibson91 ! The repo is https://mybinder.org/v2/gh/ciakovx/ciakovx.github.io/master. Another question: is the 1 GB memory limit shared between users? We aren't dealing with very big data or complex analysis, so I'm not concerned but I do want to clarify that point.
Sarah Gibson
@sgibson91
No, each session/person should get 1GB :)
InSon Zeng
@son520804
It's nice that I finally got a reasonable external ID for the binderhub! By typing the URL it at least showed up "Forbidden 403". What should be the next step to connect a Jupyterhub link/repo to that Binderhub URL?
Sarah Gibson
@sgibson91
It should all be in the Zero to BinderHub docs: https//binderhub.readthedocs.io/zero-to-binderhub
InSon Zeng
@son520804
Thank you! I found that my description was not clear. What I have done during the weekend were: 1) Redo everything in the zero-to-binderhub tutorial (originally I run the command on the local bash, I just ran all commands on Google Shell), 2) I surprising found that the external link is available, 3) I finished the tutorial until 3.7.3 (DinD)
Sarah Gibson
@sgibson91
Everything in 3.7 is optional so I'm still not quite sure what your question/problem is?
1 reply
InSon Zeng
@son520804

My problem is that if I open the link: http://<External IP>/hub/login, 403 : Forbidden will show up, instead of showing up a connected Jupyter Hub page / github repo page.

If everything looks fine, could you recommend what steps are essential in making the page set up possible?

Sarah Gibson
@sgibson91
If you're getting a 403 when trying to access the login page, then I think you've made a mistake setting up the authentication. It's hard to tell what without seeing your config file
3 replies
You may be better off opening a help question here and posting your config file (secrets redacted!): https://discourse.jupyter.org
1 reply
Loic Tetrel
@ltetrel
Hi all,
I posted a new topic on the discourse related to some issues with binderhub on a k8s cluster from opennebula.
If you have any ideas please let me know :)
https://discourse.jupyter.org/t/networking-issue-with-opennebula-k8s-service/5501