Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Pavel Tatarskiy
    @vintikzzz
    and get stuck again
    Pavel Tatarskiy
    @vintikzzz
    dashboard shows that everything ok
    image.png
    But i can't connect to it
    Pavels-iMac:frontend vintikzzzz$ kubectl top no
    Error from server (ServiceUnavailable): the server is currently unable to handle the request (get nodes.metrics.k8s.io)
    Pavel Tatarskiy
    @vintikzzz
    is it time to reset cluster?
    arduinopepe
    @arduinopepe
    Hi guys
    I'm tryng to delete a namespace but is still in hang state
    jenkins Terminating 5d22h
    is there any issu on k8s services ?
    Thomas Coudert
    @thcdrt
    Hello @vintikzzz , cheking with you in private
    Joël LE CORRE
    @jlecorre_gitlab
    Hello @arduinio there is no outage in progress at the moment.
    Maybe there are some non terminated finalizers in your namespace?
    arduinopepe
    @arduinopepe
    ok
    now i'm resolved
    thanks a lot
    Pavel Tatarskiy
    @vintikzzz
    @thcdrt all work again, thank you!
    Michał Frąckiewicz
    @SystemZ

    Is RBAC any different on OVH k8s cluster than let's say GCP?
    One of the devs from argo Slack said me this:

    I’m afraid I don’t know what could be the problem. You may need to check with OVH why this wouldnt work

    yctn
    @yctn
    @SystemZ no rbac is rbac. but how rbac itself is designed could be very differently yes
    Michał Frąckiewicz
    @SystemZ

    Ok, I'll write more details.
    There is yaml which considering other yamls should give one pod some godlike permissions on cluster:

    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: argocd-application-controller
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: argocd-application-controller
    subjects:
    - kind: ServiceAccount
      name: argocd-application-controller
      namespace: argocd

    yet, it doesn't have any:

    argocd@argocd-application-controller-5d5866cf56-8lbkd:~$ kubectl get clusterroles
    Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:argocd:argocd-application-controller" cannot list clusterroles.rbac.authorization.k8s.io at the cluster scope

    Any idea how to debug it?

    Philippe Vienne
    @PhilippeVienne_gitlab
    @SystemZ You are specifying a argocd-application-controller ClusterRole (line 8) isn't cluster-admin you want to designate ?
    Michał Frąckiewicz
    @SystemZ
    If I recall correctly, I tried it to and doesn't work either
    Let me try again...
    Philippe Vienne
    @PhilippeVienne_gitlab
    Edit your cluster role binding then recreate your pod (otherwise secret JWT is not refreshed)
    Michał Frąckiewicz
    @SystemZ
    oh, it needs restart? ok, let's try it
    Christian
    @zeeZ
    Does it really? That'd be an important detail I also didn't know
    Michał Frąckiewicz
    @SystemZ
    I removed pod, it recreated itself, still no enough permissions with cluster-admin
    argocd@argocd-application-controller-5d5866cf56-ct94d:~$ kubectl get clusterroles
    Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:argocd:argocd-application-controller" cannot list clusterroles.rbac.authorization.k8s.io at the cluster scope
    Christian
    @zeeZ
    kubectl -n argocd get serviceaccount argocd-application-controller
    kubectl describe clusterrole argocd-application-controller
    kubectl describe clusterrolebinding argocd-application-controller
    Those should work and match as a first sanity check, unless I mistyped on mobile
    Also check if that cluster-admin role really exists
    Michał Frąckiewicz
    @SystemZ
    systemz@pc:~$ kubectl -n argocd get serviceaccount argocd-application-controller
    NAME                            SECRETS   AGE
    argocd-application-controller   1         11h
    
    
    systemz@pc:~$ kubectl describe clusterrole argocd-application-controller
    Name:         argocd-application-controller
    Labels:       app.kubernetes.io/component=application-controller
                  app.kubernetes.io/name=argocd-application-controller
                  app.kubernetes.io/part-of=argocd
    Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRole","metadata":{"annotations":{},"labels":{"app.kubernetes.io/component":"ap...
    PolicyRule:
      Resources  Non-Resource URLs  Resource Names  Verbs
      ---------  -----------------  --------------  -----
      *.*        []                 []              [*]
                 [*]                []              [*]
    
    
    
    systemz@pc:~$ kubectl describe clusterrolebinding argocd-application-controller
    Name:         argocd-application-controller
    Labels:       <none>
    Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRoleBinding","metadata":{"annotations":{},"name":"argocd-application-controlle...
    Role:
      Kind:  ClusterRole
      Name:  cluster-admin
    Subjects:
      Kind            Name                           Namespace
      ----            ----                           ---------
      ServiceAccount  argocd-application-controller  argocd
    yep, it exists
    systemz@pc:~$ kubectl get clusterrole
    NAME                                                                   AGE
    admin                                                                  264d
    argocd-application-controller                                          11h
    argocd-server                                                          11h
    calico                                                                 264d
    calico-node-3.6.0                                                      173d
    cloud-controller-manager                                               264d
    cluster-admin                                                          264d
    ...
    Christian
    @zeeZ
    kubectl auth can-i list clusterroles.rbac.authorization.k8s.io
    As the Argo account. Still doesn't make any sense to me why it wouldn't work
    Michał Frąckiewicz
    @SystemZ
    systemz@pc:~$ kubectl auth can-i list clusterroles.rbac.authorization.k8s.io
    Warning: resource 'clusterroles' is not namespace scoped in group 'rbac.authorization.k8s.io'
    yes
    
    argocd@argocd-application-controller-5d5866cf56-ct94d:~$ kubectl auth can-i list clusterroles.rbac.authorization.k8s.io
    no
    Christian
    @zeeZ
    Which means there is either an absolutely stupid facepalm thing I'm missing, or your cluster is weird
    Michał Frąckiewicz
    @SystemZ
    I'm curious if I can replicate that on fresh OVH k8s cluster
    Christian
    @zeeZ
    Try kubectl auth can-i .... --as=system:serviceaccount:argocd... on your admin account, substitute accordingly
    Michał Frąckiewicz
    @SystemZ
    something like this?
    systemz@pc:~$ kubectl auth can-i list --as=system:serviceaccount:argocd-application-controller clusterroles.rbac.authorization.k8s.io
    Warning: resource 'clusterroles' is not namespace scoped in group 'rbac.authorization.k8s.io'
    no
    Christian
    @zeeZ
    Yeah. You're missing the namespace after serviceaccount: though
    sys:sa:ns:acc
    Michał Frąckiewicz
    @SystemZ

    oh man, this are long strings in a cmd :)

    kubectl auth can-i list --as=system:serviceaccount:argocd:argocd-application-controller clusterroles.rbac.authorization.k8s.io
    Warning: resource 'clusterroles' is not namespace scoped in group 'rbac.authorization.k8s.io'
    no

    still "no", though

    Michał Frąckiewicz
    @SystemZ
    Hmmm, I started new 1.11 cluster to replicate my setup and it's "yes"
    so my cluster is misconfigured somehow
    Dennis van der Veeke
    @MrDienns
    is rbac enabled on our clusters? in the kubernetes docs, it says "To enable RBAC, start the apiserver with --authorization-mode=RBAC", which I dont think we end users can do?
    Thomas Coudert
    @thcdrt
    Hello @MrDienns , you can see all features enabled here: https://docs.ovh.com/gb/en/kubernetes/exposed-apis-software-versions-reserved-resources/
    Indeed RBAC are enabled
    Dennis van der Veeke
    @MrDienns
    fantastic, thank you
    if I specify in my k8s role that that role only has access to the secrets in a specific namespace, does that mean that that role will have access to all secrets in that namespace? is there some way (if needed) of specifying that my role only has access to one particular secret, while also being in a shared namespace?
    Christian
    @zeeZ
    Yes. You can specify a resourceNames list in your roles rules