Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    bzub
    @bzub
    Did you try deleting the example objects and starting again?
    Illya Chekrygin
    @ichekrygin
    yep
    Jared Watts
    @jbw976
    hmm, is /opt/bin in $PATH?
    Illya Chekrygin
    @ichekrygin
    yes
    bzub
    @bzub
    it looked like it was in the kubelet service he posted, too
    Jared Watts
    @jbw976
    you're right @bzub i see it specifically defined in an ENVIRONMENT directive now
    is rbd fully self contained? or could it be one of its dependencies that is actually not being found
    Illya Chekrygin
    @ichekrygin
    should there be something in kubelet log?
    i just grepped for rbd and nothing
    bzub
    @bzub
    you mention minions @ichekrygin , did you install it on all kubernetes nodes inclusing masters?
    Illya Chekrygin
    @ichekrygin
    no, not the master
    master scheduling is disabled
    or should it be installed there just the same?
    bzub
    @bzub
    I'm not sure if that would applly to a PersistentVolume though, it's worth a shot
    probably not the issue though
    well actually.. it could be the issue
    the API server might need it
    Illya Chekrygin
    @ichekrygin
    ok, installing
    Illya Chekrygin
    @ichekrygin
    same thing after installing rbd on master
    bzub
    @bzub
    ok, hmm
    I see --allow-privileged=true for the kubelet, and I assume they are all like that. Double check that option is set for the API service
    Illya Chekrygin
    @ichekrygin
    ohh.. kube-api doesn't have PATH set at all..., will add it and rerun
    bzub
    @bzub
    ah good idea
    about your earlier comment, I think you should indeed see a related error in one of the kubelet logs
    Steve Leon
    @kokhang
    according to https://coreos.com/kubernetes/docs/latest/kubelet-wrapper.html, you need to have 1.3.6 or higher coreos running
    bzub
    @bzub
    he's not using kubelet-wrapper
    Steve Leon
    @kokhang
    ahh @bzub already pointed that out
    bzub
    @bzub
    apparantly kube-aws doesn't us it
    bzub
    @bzub
    @ichekrygin if that doesn't work, the only thing I can think of right now is to try installing all the Ceph clients like you did with /opt/bin/rbd
    this should do it: sudo /usr/bin/docker run --rm -v /opt/bin:/opt/bin quay.io/coffeepac/ceph-install
    actually, I wouldn't use that image
    I don't know why they have that still there, the official docker image is kept up to date, same command though:
    /usr/bin/docker run --rm -v /opt/bin:/opt/bin ceph/install-utils
    Steve Leon
    @kokhang
    @ichekrygin if you want to get started with Rook quickly, you can try to use rook's coreos K8s vagrant. https://github.com/rook/coreos-kubernetes
    This has all the modprobe volumes added
    bzub
    @bzub
    http://stackoverflow.com/a/39180049 similar issue on stackoverflow. They show their rbd script and said it worked after upgrading CoreOS. hmm
    Illya Chekrygin
    @ichekrygin
    hi, just to followup, finally got it working, needed to add rbd to the PATH in kube-controller-manager.service
    bzub
    @bzub
    wow ok. Good to know thanks!
    Bassam Tabbara
    @bassam
    Nice. Our proposed volume plugin for Rook will make sysfs calls directly and remove the need for rbd and an installation of ceph on the host/kubelet level
    Illya Chekrygin
    @ichekrygin
    totally excited, got tired of the mounting EBS nonsense
    Bassam Tabbara
    @bassam
    @ichekrygin would love to understand your scenarios of running rook in AWS. we've been assuming that most people would be using rook on-premise and not necessarily in AWS/Azure etc.
    Illya Chekrygin
    @ichekrygin
    hi, to circle back, after running Prometheus w/ Rook on K8S for last 12 hrs
    blob
    last gap, is me forcing the Prometheus pod to start on another EC2 instance
    In comparison to:
    blob
    Running Prometheus w/ EBS on K8S: first gap EC2 with Prometheus pod is being killed by AWS, second: Prometheus pod restart
    Bassam Tabbara
    @bassam
    whats the Y axis?