Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Joël LE CORRE
    @jlecorre_gitlab
    Hello @arduinio there is no outage in progress at the moment.
    Maybe there are some non terminated finalizers in your namespace?
    arduinopepe
    @arduinopepe
    ok
    now i'm resolved
    thanks a lot
    Pavel Tatarskiy
    @vintikzzz
    @thcdrt all work again, thank you!
    Michał Frąckiewicz
    @SystemZ

    Is RBAC any different on OVH k8s cluster than let's say GCP?
    One of the devs from argo Slack said me this:

    I’m afraid I don’t know what could be the problem. You may need to check with OVH why this wouldnt work

    yctn
    @yctn
    @SystemZ no rbac is rbac. but how rbac itself is designed could be very differently yes
    Michał Frąckiewicz
    @SystemZ

    Ok, I'll write more details.
    There is yaml which considering other yamls should give one pod some godlike permissions on cluster:

    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: argocd-application-controller
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: argocd-application-controller
    subjects:
    - kind: ServiceAccount
      name: argocd-application-controller
      namespace: argocd

    yet, it doesn't have any:

    argocd@argocd-application-controller-5d5866cf56-8lbkd:~$ kubectl get clusterroles
    Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:argocd:argocd-application-controller" cannot list clusterroles.rbac.authorization.k8s.io at the cluster scope

    Any idea how to debug it?

    Philippe Vienne
    @PhilippeVienne_gitlab
    @SystemZ You are specifying a argocd-application-controller ClusterRole (line 8) isn't cluster-admin you want to designate ?
    Michał Frąckiewicz
    @SystemZ
    If I recall correctly, I tried it to and doesn't work either
    Let me try again...
    Philippe Vienne
    @PhilippeVienne_gitlab
    Edit your cluster role binding then recreate your pod (otherwise secret JWT is not refreshed)
    Michał Frąckiewicz
    @SystemZ
    oh, it needs restart? ok, let's try it
    Christian
    @zeeZ
    Does it really? That'd be an important detail I also didn't know
    Michał Frąckiewicz
    @SystemZ
    I removed pod, it recreated itself, still no enough permissions with cluster-admin
    argocd@argocd-application-controller-5d5866cf56-ct94d:~$ kubectl get clusterroles
    Error from server (Forbidden): clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:argocd:argocd-application-controller" cannot list clusterroles.rbac.authorization.k8s.io at the cluster scope
    Christian
    @zeeZ
    kubectl -n argocd get serviceaccount argocd-application-controller
    kubectl describe clusterrole argocd-application-controller
    kubectl describe clusterrolebinding argocd-application-controller
    Those should work and match as a first sanity check, unless I mistyped on mobile
    Also check if that cluster-admin role really exists
    Michał Frąckiewicz
    @SystemZ
    systemz@pc:~$ kubectl -n argocd get serviceaccount argocd-application-controller
    NAME                            SECRETS   AGE
    argocd-application-controller   1         11h
    
    
    systemz@pc:~$ kubectl describe clusterrole argocd-application-controller
    Name:         argocd-application-controller
    Labels:       app.kubernetes.io/component=application-controller
                  app.kubernetes.io/name=argocd-application-controller
                  app.kubernetes.io/part-of=argocd
    Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRole","metadata":{"annotations":{},"labels":{"app.kubernetes.io/component":"ap...
    PolicyRule:
      Resources  Non-Resource URLs  Resource Names  Verbs
      ---------  -----------------  --------------  -----
      *.*        []                 []              [*]
                 [*]                []              [*]
    
    
    
    systemz@pc:~$ kubectl describe clusterrolebinding argocd-application-controller
    Name:         argocd-application-controller
    Labels:       <none>
    Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"rbac.authorization.k8s.io/v1","kind":"ClusterRoleBinding","metadata":{"annotations":{},"name":"argocd-application-controlle...
    Role:
      Kind:  ClusterRole
      Name:  cluster-admin
    Subjects:
      Kind            Name                           Namespace
      ----            ----                           ---------
      ServiceAccount  argocd-application-controller  argocd
    yep, it exists
    systemz@pc:~$ kubectl get clusterrole
    NAME                                                                   AGE
    admin                                                                  264d
    argocd-application-controller                                          11h
    argocd-server                                                          11h
    calico                                                                 264d
    calico-node-3.6.0                                                      173d
    cloud-controller-manager                                               264d
    cluster-admin                                                          264d
    ...
    Christian
    @zeeZ
    kubectl auth can-i list clusterroles.rbac.authorization.k8s.io
    As the Argo account. Still doesn't make any sense to me why it wouldn't work
    Michał Frąckiewicz
    @SystemZ
    systemz@pc:~$ kubectl auth can-i list clusterroles.rbac.authorization.k8s.io
    Warning: resource 'clusterroles' is not namespace scoped in group 'rbac.authorization.k8s.io'
    yes
    
    argocd@argocd-application-controller-5d5866cf56-ct94d:~$ kubectl auth can-i list clusterroles.rbac.authorization.k8s.io
    no
    Christian
    @zeeZ
    Which means there is either an absolutely stupid facepalm thing I'm missing, or your cluster is weird
    Michał Frąckiewicz
    @SystemZ
    I'm curious if I can replicate that on fresh OVH k8s cluster
    Christian
    @zeeZ
    Try kubectl auth can-i .... --as=system:serviceaccount:argocd... on your admin account, substitute accordingly
    Michał Frąckiewicz
    @SystemZ
    something like this?
    systemz@pc:~$ kubectl auth can-i list --as=system:serviceaccount:argocd-application-controller clusterroles.rbac.authorization.k8s.io
    Warning: resource 'clusterroles' is not namespace scoped in group 'rbac.authorization.k8s.io'
    no
    Christian
    @zeeZ
    Yeah. You're missing the namespace after serviceaccount: though
    sys:sa:ns:acc
    Michał Frąckiewicz
    @SystemZ

    oh man, this are long strings in a cmd :)

    kubectl auth can-i list --as=system:serviceaccount:argocd:argocd-application-controller clusterroles.rbac.authorization.k8s.io
    Warning: resource 'clusterroles' is not namespace scoped in group 'rbac.authorization.k8s.io'
    no

    still "no", though

    Michał Frąckiewicz
    @SystemZ
    Hmmm, I started new 1.11 cluster to replicate my setup and it's "yes"
    so my cluster is misconfigured somehow
    Dennis van der Veeke
    @MrDienns
    is rbac enabled on our clusters? in the kubernetes docs, it says "To enable RBAC, start the apiserver with --authorization-mode=RBAC", which I dont think we end users can do?
    Thomas Coudert
    @thcdrt
    Hello @MrDienns , you can see all features enabled here: https://docs.ovh.com/gb/en/kubernetes/exposed-apis-software-versions-reserved-resources/
    Indeed RBAC are enabled
    Dennis van der Veeke
    @MrDienns
    fantastic, thank you
    if I specify in my k8s role that that role only has access to the secrets in a specific namespace, does that mean that that role will have access to all secrets in that namespace? is there some way (if needed) of specifying that my role only has access to one particular secret, while also being in a shared namespace?
    Christian
    @zeeZ
    Yes. You can specify a resourceNames list in your roles rules
    Dennis van der Veeke
    @MrDienns
    thank you very much, i shall try that out later
    Christian
    @zeeZ
    Official rbac doc has it in an example. Not that it doesn't work for all verbs (or didn't last I checked), such as list
    Dennis van der Veeke
    @MrDienns
    as long as I can prevent a pod from reading any other secret than the one it needs, it's good :)
    Michał Frąckiewicz
    @SystemZ
    How long does adding smallest B2-7 node to k8s cluster can take?
    I think it's now more than 20 mins in "Installing" state
    Thomas Coudert
    @thcdrt
    It should take several minutes
    More than 10 mn it begins to not be normal
    Michał Frąckiewicz
    @SystemZ
    I'll give info in another 10mins, just to be sure
    Thomas Coudert
    @thcdrt
    Can you send me your cluster id in private please ?
    Michał Frąckiewicz
    @SystemZ
    ok
    Michał Frąckiewicz
    @SystemZ
    @thcdrt thx, it's working now :)