EnMasseProject/community

0.31.0 RC3 is out: https://github.com/EnMasseProject/enmasse/releases/tag/0.31.0-rc3

Bob Claerhout

@BobClaerhout

Ok, this is unfortunate. Are there any plans/ideas/... to remove the requirement of being a cluster admin?

Ulf Lilleengen

@lulf

@BobClaerhout for a 'bundle' install, cluster-admin will probably required as long as the CRDs require it, and also some cluster-wide privileges for the operators to manage the CRD instances and for the console. However, with OpenShift 4, EnMasse is available in the operator hub: https://operatorhub.io/operator/enmasse . It might be that if ARHO provides OpenShift 4 clusters, it might allow for installing EnMasse that way.

Bob Claerhout

@BobClaerhout

Ok, thanks for the follow up

Dirk Van Haerenborgh

@vhdirk

Hey guys. I've been busy deploying enmasse on an AKS cluster, following the instructions here https://enmasse.io/documentation/master/kubernetes/

I've been successful in creating addressspaces, addressspace plans, addresses, etc

With clients, I can subscribe to topics, so that's all good

However, when running kubectl describe addressspace <myspace>, after 5 hours it is still saying: "Is Ready: false"

and: Messages: Endpoint 'console' is not yet exposed

Do you guys know of anything I might have missed?

k-wall

@k-wall

Hi Dirk - the output of kubectl get pods -n enmasse-infra might help. Also check the status section of the addressspace resource you have create (kubectl get addressspace <myspace> -o yaml). Which version of EnMasse is this btw

Dirk Van Haerenborgh

@vhdirk

@k-wall thanks for the quick reply. This is enmasse 0.30.2

Anything in particular I'm looking for in the output of get addressspace? It ends on:

  isReady: false
  messages:
  - Endpoint 'console' is not yet exposed
  phase: Configuring

k-wall

@k-wall

@vhdirk what do the pods look like?

Dirk Van Haerenborgh

@vhdirk

also, the externalPorts fields of endpointStatuses are empty

well, all pods are running :) with 0 restarts for the last 5 hours

(expect for the times where I decided to explicitly delete some pods in hopes to see some meaningful info)

NAME                                             READY   STATUS    RESTARTS   AGE
address-space-controller-895cf7485-bzvx2         1/1     Running   0          30m
admin.aloxyio-5796c45855-7m8wj                   2/2     Running   0          6h6m
api-server-d6fc477d8-r69vw                       1/1     Running   0          6h12m
broker-aloxyio-dpx0-0                            1/1     Running   0          6h5m
enmasse-operator-575799b4b6-w7rgx                1/1     Running   0          6h12m
qdrouterd-aloxyio-0                              1/1     Running   2          6h6m
qdrouterd-aloxyio-1                              1/1     Running   0          6h4m
standard-authservice-59956d67c-zp5bv             1/1     Running   0          6h9m

(my addresspace is called 'aloxyio)

k-wall

@k-wall

@vhdirk check the log of the address-space-controller for exceptions. It is it that is response for updating the status of the addressspace resource.

if there is nothing confidential in the log and can share it (pastebin etc), even better

Dirk Van Haerenborgh

@vhdirk

this is the full output of the addressspace : https://gist.github.com/vhdirk/f41b3d89aa6e607726fe549ef15d2211

I'm running everything inside the 'aloxy' namespace, not the default enmasse-infra namespace

k-wall

@k-wall

@vhdirk I meant the address-space-controller-895cf7485-bzvx2 log rather than the yaml.. kubectl log address-space-controller-895cf7485-bzvx2 etc

Dirk Van Haerenborgh

@vhdirk

I know :) here it is: https://gist.github.com/vhdirk/8ad2ef2ed9fa618d0dfefaf163391a16

k-wall

@k-wall

Ulf Lilleengen

@lulf

Console seems to have endpoint exposed as type route , which is not a supported type on kubernetes. ( looking at address space yaml)

k-wall

@k-wall

yes, I'd just noticed that myself

@vhdirk what did your original addressspace yaml look like?

Dirk Van Haerenborgh

@vhdirk

ah yes, that's indeed rather stupid:

kind: AddressSpace
apiVersion: enmasse.io/v1beta1
metadata:
  annotations:
    enmasse.io/infra-uuid: aloxyio
  name: aloxyio
spec:
  type: standard
  plan: standard-unlimited
#  authenticationService:
#    type: standard
  endpoints:
    - name: messaging
      service: messaging
      cert:
        provider: selfsigned
    - name: mqtt
      service: mqtt
      expose:
        type: loadbalancer
        loadBalancerPorts:
        - mqtt
        - mqtts
      cert:
        provider: selfsigned
    - name: console
      service: console
      expose:
        type: route
        routeTlsTermination: reencrypt
        routeServicePort: https
      cert:
        provider: selfsigned

I started off from an openshift project that now has to run on AKS.

for k8s, that has to be loadbalancer I suppose?

k-wall

@k-wall

@vhdirk yes

Dirk Van Haerenborgh

@vhdirk

haha, awesome. thanks!

k-wall

@k-wall

no problem

Dirk Van Haerenborgh

@vhdirk

ok, so now the addresspace is reported to be ready

but then how do I access the console?

Dirk Van Haerenborgh

@vhdirk

a port forward to svc/console-aloxyio:8081 just results in connection refused by the pod

k-wall

@k-wall

@vhdirk on kubernetes getting the console is a bit more involved. the process is documented https://enmasse.io/documentation/master/kubernetes/#config-openid-connect-for-kubernetes I haven't tried this with AKS myself.

Dirk Van Haerenborgh

@vhdirk

Oh, thanks. must've missed that bit. I'll have to figure out how to use keycloak for oath then.

Dirk Van Haerenborgh

@vhdirk

Hi @k-wall Apparently, you can't configure an AKS cluster to use OpenID Connect. Is there any other way to access the console? or something similar?

Ulf Lilleengen

@lulf

@vhdirk I'm not aware of any way to configure OIDC with any managed Kubernetes provider I'm afraid. The console relies on Kubernetes knowing the identity of the users tied with the RBAC roles to do authorization. I'm not sure how easy it would be to provide a customizable auth/authz to the console at this point.

Ulf Lilleengen

@lulf

@k-wall Am I right in thinking that creating an access controller in pkg/consolegraphql/accesscontroller that uses something other than Kubernetes RBAC would allow to use the console with an OIDC provider without requiring the Kubernetes cluster itself to be configured with OIDC?

k-wall

@k-wall

@lulf no - that wouldn't resolve the whole issue. for creates/update/delete, the console just invokes the kubernetes API passing the token of the of the logged in user. We rely on oauth-proxy to make that token available.

Ulf Lilleengen

@lulf

@k-wall hmm, ok... i was also thinking about this in the context of the sandbox.enmasse.io that I'd like to set up. I was hoping to use some managed Kubernetes provider for that, but I think maybe some cloud VM with Kubernetes + Keycloak for OIDC is probably needed then.

k-wall

@k-wall

@lulf yeah - I don't see a way to side step the needs for kubernetes to be integrated with an external open-id service without resorting by managing our own auth/authz. Keycloak seems like the path of least resistance to me.

(I was thinking about improving the SmokeTests on kubernetes to include keycloak so that the console could be minimally tested on that platform)

Ulf Lilleengen

@lulf

I wonder why the providers are not providing capability to configure OIDC

k-wall

@k-wall

@lulf agreed, that does seem odd to me.

Ulf Lilleengen

@lulf

@vhdirk you might want to have a look at https://blog.jetstack.io/blog/kube-oidc-proxy/ . It should be possible to point the enmasse console to this proxy and have it configured with the OIDC provider of your choice. I’m looking at this in the context of deploying an enmasse sandbox (for new users evaluating enmasse), so i will try it.

Dirk Van Haerenborgh

@vhdirk

@lulf thanks, I'll take a look!

Where communities thrive

People

Repo info

Activity