Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Frederic
    @FredericLatour
    No way to have any statement/answer from OVH on the important topic of LB following my previous post ? If I missed any information besides the email we received, please provide some links to any additional documentation. Thanks.
    Adrien Ferrand
    @adferrand
    Ok guys, this is not possible! On top of my problem with Role/RoleBinding, I just lost one of my node, breaking several systems in production!
    This problem is here for months, and it is worsening: the nodes cannot sustain any CPU spike. As soon as there is a little too much CPU load, the kubelet daemon is breaking and the node is leaving the cluster with all the pods active on it.
    I have 16 nodes that cover 10 times (in term of CPU) and 3 times (in term of RAM) the average load. This should be sufficient. It is normally planned to modify node configuration to put highest priority to the kubelet, when it will be done ?
    And why failed node are not cleaned up / restarted automatically ? I need to watch it and restart manually each time
    vkukk
    @vkukk

    We still have constant and unnecessary resize events in our cluster: 15588 Resizing: External resizer is resizing volume pvc-87468728-9df0-43aa-b5c4-aa6a0e2264e5

    Events emitted by the external-resizer cinder.csi.openstack.org seen at 2021-01-15 14:56:31 +0000 UTC since 2020-12-10 11:09:24 +0000 UTC

    1 reply
    Maxime Hurtrel
    @crazyman_twitter

    Hello @all !

    After a couple of months of private beta, the vRack is now available to everyone (in Public Beta)
    More details and exhaustive documentation is available here : https://github.com/ovh/public-cloud-roadmap/issues/15#issuecomment-761001758

    The feature will graduate to GA late February/ early March (as soon as control panel and LBaaS public IP to private backend are done. (Both are being developed as we speak ).

    tsn77130
    @tsn77130
    hi @crazyman_twitter , thanks for vrack update, that's a nice feature, but for now, above all we need to have some clarifications about iplb migration of monday.
    Do you have some informations for us (there is some unresolved questions in this thread)
    thanks
    2 replies
    Adrien Ferrand
    @adferrand
    A third problem, for the road: my ingress controllers fail to retrieve the routing configuration because of this error Failed to list *v1.Endpoints: Get "https://10.3.0.1:443/api/v1/endpoints?resourceVersion=30284268222": dial tcp 10.3.0.1:443: connect: connection refused
    Seems that the API server is down in some way
    Adrien Ferrand
    @adferrand
    And on the loadbalancers, this error: Error updating Endpoint Slices for Service ingress-controllers/restricted-traefik: failed to update restricted-traefik-dsfcz EndpointSlice for Service ingress-controllers/restricted-traefik: Operation cannot be fulfilled on endpointslices.discovery.k8s.io "restricted-traefik-dsfcz": the object has been modified; please apply your changes to the latest version and try again
    korioz
    @korioz:matrix.org
    [m]

    Hi, pretty new in these kubernetes stuff followed the dashboard setup but i run into

    secrets is forbidden: User "system:serviceaccount:kubernetes-dashboard:admin-user" cannot list resource "secrets" in API group "" in the namespace "default"

    cluster version 1.17

    balasu
    @balasu_gitlab
    Hi @korioz:matrix.org , instead of default dashboard, pl try kubectl create namespace portainer && kubectl apply -n portainer -f https://raw.githubusercontent.com/portainer/k8s/master/deploy/manifests/portainer/portainer.yaml you could open the UI n node port 30777 https://documentation.portainer.io/v2.0/deploy/linux/
    korioz
    @korioz:matrix.org
    [m]
    reinstalled the cluster already fixed the problem still weird, thx for the help @balasu_gitlab
    korioz
    @korioz:matrix.org
    [m]
    So in ovh you can't do external udp load balancing?
    2 replies
    meh :/
    Alexandre Ellert
    @aellert_twitter
    Hi, I'd like to run a specific workload which need high CPU ressources on a specific nodepool. And I don't want other pods from general workload to consume ressource on the high CPU nodepool.
    From what I've read, the way to go is to combine "taint" and "tolerations"
    Alexandre Ellert
    @aellert_twitter
    What I want is when a new node integrates the high CPU nodepool, the new node should be be tainted automatically.
    Is there a way to do that ?
    5 replies
    Thank you !
    Frédéric MARTIN
    @fmartin-linagora
    Hi,
    Since last friday, my cluster is not able to attach volumes, configmaps, secrets or tokens.
    I have the following error: attachment timeout for volume or unattached volumes ... : timed out waiting for the condition.
    Does any one have the same problem ?
    2 replies
    Waseem Awashra
    @wawashra
    hello, im use this chart for MongoDB https://github.com/bitnami/charts/tree/master/bitnami/mongodb
    but always i have a MongoError: not master and slaveOk=false error on my applications
    what the best way to solve this problem
    Philippe Vienne
    @PhilippeVienne_gitlab
    Hello, we have a strage behaviour on the LB upgrade, an old cluster keep having CNAME LB and a newer one got an IP LB. Anyone from OVH side to understand the issue?
    Sebert
    @asebert_gitlab
    Hello, is the private registry down ?? Error response from daemon: Get https://zs35a00n.gra7.container-registry.ovh.net/v2/: received unexpected HTTP status: 503 Service Temporarily Unavailable
    4 replies
    Chaya56
    @Chaya56
    Hello
    we got again a node failing
    chaya@DESKTOP-BHGN56S  /mnt/c/Data/Projects/Origination-ops/k8s/help/proxy-local-to-cluster   master ±  kubectl -n prod get node
    NAME STATUS ROLES AGE VERSION
    node-1-cpu Ready <none> 183d v1.18.1
    node-2-cpu Ready <none> 181d v1.18.1
    node-3-ram Ready <none> 181d v1.18.1
    node-4-ram Ready <none> 181d v1.18.1
    node-5-ram Ready <none> 181d v1.18.1
    node-ce60a59e-cc49-478d-8824-b32435e40ca1 NotReady <none> 119d v1.18.6
    chaya@DESKTOP-BHGN56S  /mnt/c/Data/Projects/Origination-ops/k8s/help/proxy-local-to-cluster   master ±  kubectl -n prod top node
    NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
    node-1-cpu 2510m 31% 20321Mi 72%
    node-2-cpu 3817m 48% 14889Mi 53%
    node-3-ram 699m 36% 17955Mi 64%
    node-4-ram 951m 50% 15763Mi 56%
    node-5-ram 629m 33% 18717Mi 67%
    node-ce60a59e-cc49-478d-8824-b32435e40ca1 <unknown> <unknown> <unknown> <unknown>
    Chaya56
    @Chaya56

    Events:
    Type Reason Age From Message


    Normal NodeHasNoDiskPressure 16m (x3 over 7d18h) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeHasNoDiskPressure
    Normal NodeHasSufficientPID 16m (x3 over 7d18h) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeHasSufficientPID
    Normal NodeNotReady 16m (x2 over 17m) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeNotReady
    Normal NodeHasSufficientMemory 16m (x3 over 7d18h) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeHasSufficientMemory
    Normal NodeNotReady 10m kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeNotReady
    Normal Starting 10m kubelet Starting kubelet.
    Normal NodeHasSufficientMemory 10m (x2 over 10m) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeHasSufficientMemory
    Normal NodeHasNoDiskPressure 10m (x2 over 10m) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeHasNoDiskPressure
    Normal NodeHasSufficientPID 10m (x2 over 10m) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeHasSufficientPID
    Normal NodeAllocatableEnforced 10m kubelet Updated limits on kube reserved cgroup /system.slice
    Normal NodeAllocatableEnforced 10m kubelet Updated Node Allocatable limit across pods
    Normal NodeReady 10m kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeReady
    Normal NodeHasSufficientMemory 3m40s (x8 over 3m53s) kubelet Node node-ce60a59e-cc49-478d-8824-b32435e40ca1 status is now: NodeHasSufficientMemory

    Chaya56
    @Chaya56
    always the same node
    Frederic
    @FredericLatour

    Isn't there anyone from OVH that can help here??????
    Must not be that difficult.
    Here is what I was using for helm ingress config (was based on a gist that was published by someone from OVH if I can remember well)

    controller:
      service:
        externalTrafficPolicy: "Local"
        annotations: {"service.beta.kubernetes.io/ovh-loadbalancer-proxy-protocol": "v1"}
      config:
        use-proxy-protocol: "true"
        proxy-real-ip-cidr: "10.108.0.0/14"
        use-forwarded-headers: "false"
        http-snippet: |
          geo $realip_remote_addr $is_lb {
            default       0;
            10.108.0.0/14 1;
          }
        server-snippet: |
          if ($is_lb != 1) {
            return 403;
          }

    How do I need to change this for the production LB ?

    Please make an effort and answer this question.
    Thansk in advance

    4 replies
    Chaya56
    @Chaya56
    Hello @FredericLatour have you tried to replace 10.108.0.0 by your nginx ingress loadbalancer ?
    Chaya56
    @Chaya56
    you have to whitelise internal and external IP of your loadbalancer, (in case of loop traffic, for http01 challenge by example)
    Frederic
    @FredericLatour

    @Chaya56 Unfortunately, what you are saying is not clear at all for me.
    What should I change?
    proxy-real-ip-cidr?
    http-snippet?

    With what values?
    If those values are specific to each LB. How do I get those information concretely.

    I have to whitelist internal and external IP of your loadbalancer. Where ? How ?

    Nobody from OVH anymore on this channel?

    Chaya56
    @Chaya56
    both with value of your service ip for the nginx ingress controller
    This channel is just for chatting and helping, ovh guys are here just as 'volunteer', on the other hand you can call official OVH support https://help.ovhcloud.com/fr/ (bottom of page)
    Nicolas Antoniazzi
    @nantoniazzi
    Hi, I have 4 nodes which are stuck. (I tries to reboot them, but id did not change anything). They are in a NotReady state.
    The node description says: kubelet wanted to free 10369671987 bytes, but freed 0 bytes space with errors in image deletion: [rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "registry.kubernatine.ovh/public/csi-node-driver-registrar:v1.1.0" (must force) - container 0665581f7a59 is using its referenced image a93898755322, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "registry.kubernatine.ovh/public/csi-cinder-plugin:v0.1.0" (must force) - container f2975c65cb7b is using its referenced image 25beabf0a35a]
    2 replies
    Frederic
    @FredericLatour
    @Chaya56
    Well, it's not really just for chatting ... In various occasions people are exposing their problems and people from OVH are asking their id and all that jazz in order to provide help .
    I find it somewhat disturbing that certainly various OVH guys are just reading my messages and just let me fighting with this when it would certainly be easy to clarify things.
    It's not like OVH documentation and guides were top notch .
    112 replies
    Nicolas Antoniazzi
    @nantoniazzi
    Sorry, I'm still stuck with my problem: kubelet wanted to free 10369671987 bytes, but freed 0 bytes space with errors in image deletion: [rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "registry.kubernatine.ovh/public/csi-node-driver-registrar:v1.1.0" (must force) - container 0665581f7a59 is using its referenced image a93898755322, rpc error: code = Unknown desc = Error response from daemon: conflict: unable to remove repository reference "registry.kubernatine.ovh/public/csi-cinder-plugin:v0.1.0" (must force) - container f2975c65cb7b is using its referenced image 25beabf0a35a] on 4 of my production nodes
    Can someone help me to restart those nodes or disconnect/reconnect cinder on them (I do not get what is the problem)
    Joël LE CORRE
    @jlecorre_gitlab
    Hello @nantoniazzi
    Could you send me your cluster ID in private please?
    Andy Tan
    @tanandy
    Hi, guys do you know where can i get the official communication about old LB release in march ??? i guess our hosting team didnt notice the email ??

    Hi, guys do you know where can i get the official communication about old LB release in march ??? i guess our hosting team didnt notice the email ??

    what i read on gitter is we will live with 2 LB during a period, then after some time the previous LB will be deleted, so if we dont react and update DNS to new IP we will experience downtime. Did i understand well ?

    9 replies
    yctn
    @yctn
    is the cloud provider for k8s already released yet? i run a k8s cluster on bare metal that i have build myself.
    19 replies
    Philippe Vienne
    @PhilippeVienne_gitlab
    Hello, does new LB from OVH have timeout on connections ?
    5 replies
    Frederic
    @FredericLatour

    Could someone from OVH clarify the following:

    • 2 new LB controllers were automagically created on monday (as expected) because I didn't do anything regarding the 2 LB were were using.
    • Now, when I look at my Ingress services, one display a new IP V4 address as the External IP field (sounds conform to the email we received), the other one however displays a domain name (strangely enough the domain name has the new ip as part of it's name - the IP that is displayed in the control panel associated to this LB).
    • Now I deployed "mendhak/http-https-echo" and used the first Ingress (the one with the IPV4 external IP) to access the service and it works for both the new and former IP.
    • However when using the 2nd Ingress, it's only working with the former IP and not the new IP (the one that is displayed in the control panel). By not working, I mean that when I "curl" echo service through the new IP (a domain name that points to the new ip to be precise), I get a 403 forbidden.

    My questions:

    • Did one of the LB did not initialize properly?
    • How does the compatibility work exactly? Are the new LBs configured to accept both the new and the former IP?

    Thanks in advance

    3 replies
    Oyabi
    @Oyabi
    Hi, I'm playing with ovh kubernetes and trying to install ingress.
    I am following this tuto: https://docs.ovh.com/gb/en/kubernetes/installing-nginx-ingress/
    After the creation of my service, in EXTERNAL-IP I've got and IP address and not a dns name. In the next step, I need a domain name for installing ingress. How can I get a domain name and not an IP address please?
    10 replies
    Frederic
    @FredericLatour
    OVH, any chance to get an answer to my previous post? Or in case this is too long to read, can you let me know how it is possible that one of the LB that was automatically created on Monday in replacement of the free LB I was previously using, does not have an IP address displayed as the External-IP. Instead, I get something like this:
    ip-51-xxx-xx-xx.bhs.lb.ovh.net
    3 replies
    Frederic
    @FredericLatour
    Does anybody know how to send a private message to someone that replied in a thread? Is it only me or the option for replying privately is not available when passing the mouse over the picture in a thread.
    Any alternative ?
    Simon Guyennet
    @sguyennet
    @FredericLatour