These are chat archives for spring-cloud/spring-cloud

30th
Jun 2016
arca1n
@arca1n
Jun 30 2016 00:37
@dsyer thank you for the response. spring-cloud-stream has the redis binder available and I was just trying it out.
bitsofinfo
@bitsofinfo
Jun 30 2016 00:53
@dsyer thanks man. I reviewed and ran the workaround. it does indeed work
I'm going to use this in my real project until a real fix can be worked in for spring-cloud/spring-cloud-config#434
Dave Syer
@dsyer
Jun 30 2016 03:04
Cool
bitsofinfo
@bitsofinfo
Jun 30 2016 03:05
now moving on to discovery... :)
can a eureka server, be setup to pull its config from a config server? would that even be recommended? seems like maybe not?
Spencer Gibb
@spencergibb
Jun 30 2016 03:06
yup
everything is a spring boot app :-)
bitsofinfo
@bitsofinfo
Jun 30 2016 03:07
yeah, digging what you guys have done w/ the cloud project
Dave Syer
@dsyer
Jun 30 2016 03:07
I prefer to make config server the lynchpin
bitsofinfo
@bitsofinfo
Jun 30 2016 03:07
what do you guys recommend topology wise if the config server is sort of a SPOF, obviously a lb of sorts in front of those instances?
Dave Syer
@dsyer
Jun 30 2016 03:08
But you can do it the other way (it's a bit more fragile because of lifecycle issues)
The SPOF is really only git (once you have a load balancer)
bitsofinfo
@bitsofinfo
Jun 30 2016 03:08
true
Dave Syer
@dsyer
Jun 30 2016 03:08
and config servers cache the git repo
Spencer Gibb
@spencergibb
Jun 30 2016 03:08
yes, it’s what we do in CF (re: load balancer). There’s a request to set a comma separated list in each client
Dave Syer
@dsyer
Jun 30 2016 03:08
so it's fairly robust
I'm resisting the client-side bit
(quietly)
Stephen Gream
@ThreeToes
Jun 30 2016 03:09
I've still got to figure out how to use microservices properly, I get the feeling my toy projects are too chatty
Spencer Gibb
@spencergibb
Jun 30 2016 03:09
:-) it’s easier now that you can set a server list class via properties. That way it could be done via rest template and ribbon.
bitsofinfo
@bitsofinfo
Jun 30 2016 03:10
you mean, config servers registering w/ eureka and clients finding CS via eureka, then using client side LB to deal w/ outages? vs a true haproxy/nginx/elb middle LB?
Spencer Gibb
@spencergibb
Jun 30 2016 03:10
resisting the comma separate list of config servers
Dave Syer
@dsyer
Jun 30 2016 03:10
Easier to resist? Yes.
Spencer Gibb
@spencergibb
Jun 30 2016 03:11
:laughing:
Dave Syer
@dsyer
Jun 30 2016 03:11
@bitsofinfo that seems to be the gist of what they want
(a vocal minority)
bitsofinfo
@bitsofinfo
Jun 30 2016 03:12
@dsyer you mean clients of CF want app rest client self-load balancing vs centralized? is that how CF is currently setup? (the latter, centralized)
side note: you guys have any articles on spring-cloud topologies in docker containers etc? i.e. containers up/down overlay net/discovery issues w/ orchestration (i.e. swarm etc) etc. I just know this makes it even more complex
Dave Syer
@dsyer
Jun 30 2016 03:13
We're not really very docker savvy here (at least I'm not).
Spencer has more fu than me
Spencer Gibb
@spencergibb
Jun 30 2016 03:13
:+1:
Dave Syer
@dsyer
Jun 30 2016 03:13
But really, containers are not your friend, as a developer.
They fight you the whole way
Spencer Gibb
@spencergibb
Jun 30 2016 03:13
no, that :+1: was for not savvy
Dave Syer
@dsyer
Jun 30 2016 03:13
I use docker a lot
Spencer Gibb
@spencergibb
Jun 30 2016 03:14
Let’s pass the buck to @marcingrzejszczak for docker
Dave Syer
@dsyer
Jun 30 2016 03:14
Yeah, while he's asleep
Spencer Gibb
@spencergibb
Jun 30 2016 03:14
exactly
Dave Syer
@dsyer
Jun 30 2016 03:14
He certainly has some war stories
bitsofinfo
@bitsofinfo
Jun 30 2016 03:14
well, yeah.. i mean if your app has to have peer awareness for this kind of stuff, yeah it becomes a pain as the contain has no insight natively as to "who am i ip/port), we use registrator/consul etc to do such discovery in a library I can drop in a jvm etc
Dave Syer
@dsyer
Jun 30 2016 03:14
I use docker a lot. But I wouldn't want to rely on it for deploying apps I work on.
Stephen Gream
@ThreeToes
Jun 30 2016 03:15
@dsyer What do you mean?
bitsofinfo
@bitsofinfo
Jun 30 2016 03:15
well this thing i'm building now using spring-cloud.... will be containerized. so this will be interesting
Dave Syer
@dsyer
Jun 30 2016 03:15
I use it for the local development process. Deploying middleware.
I don't use it for deploying apps. I want a platform to do that for me.
The unit of currency is too close to the metal with container still for me (as an app developer)
Tools can help, but fundamentally, it's just overhead, and I don't need it
Spencer Gibb
@spencergibb
Jun 30 2016 03:16
I agree with dave. But, if you’re already using consul have you looked at https://github.com/spring-cloud/spring-cloud-consul
Dave Syer
@dsyer
Jun 30 2016 03:16
I want my apps to run in a container, but I never want to know how that happens
If you don't have a platform, you have to use containers.
I'm 100% behind that
But I don't want to shave that yak, personally, on my own
bitsofinfo
@bitsofinfo
Jun 30 2016 03:18
yeah I read about your consul support. We have our current prototypes built around consul and registrator... but that ballgame is changing radically w/ swarm in docker 1.12....
Dave Syer
@dsyer
Jun 30 2016 03:18
Indeed
bitsofinfo
@bitsofinfo
Jun 30 2016 03:19
i.e. a lot of 3rd party tooling around container discovery is now going to just be baked in along w/ lb'ing on each swarm node itself, fully auto magical.. :) but still not sure on app -> app discovery and peer cluster formation. i.e. what that api/metadata repo will look like
Stephen Gream
@ThreeToes
Jun 30 2016 03:20
I'm just kind of curious. We have a big monolithic console application that I'd like to convert to microservices, knowing about some deployment technologies beyond our CI server would help me get some traction for the idea
Not trying to pick a fight or stir up holy wars over docker v platforms :P
bitsofinfo
@bitsofinfo
Jun 30 2016 03:21
@threetoes well thats a big question
Dave Syer
@dsyer
Jun 30 2016 03:21
@bitsofinfo it's all very interesting. I also think client side load balancing probably has a place, even in that world.
Stephen Gream
@ThreeToes
Jun 30 2016 03:21
@bitsofinfo Yeah, hence why I need to get started learning now :)
Dave Syer
@dsyer
Jun 30 2016 03:21
You can make more informed decisions on a more granular level
bitsofinfo
@bitsofinfo
Jun 30 2016 03:21
i personally like that concept of peer level shared responsibility and less reliance on "classes" of functionality that create spofs and bottleneck tiers
Dave Syer
@dsyer
Jun 30 2016 03:22
Magic infrastructure is fantastic, and we all need some of that. But the rubber hits the road in the application tier, now and always.
bitsofinfo
@bitsofinfo
Jun 30 2016 03:22
but.. too much of it can lead to bloated black boxes
true
Dave Syer
@dsyer
Jun 30 2016 03:24
For me, as a developer, a lot of it comes down to how close I feel my laptop environment is to "real". I need to run integration tests, and debug line by line. No-one has come up with a way to make that work well when my code runs remotely.
Stephen Gream
@ThreeToes
Jun 30 2016 03:24
True, I just had to debug something by using ssh port forwarding to the dev server
bitsofinfo
@bitsofinfo
Jun 30 2016 03:25
true
Stephen Gream
@ThreeToes
Jun 30 2016 03:25
That was a pain in and of itself to get going
Dave Syer
@dsyer
Jun 30 2016 03:25
You shouldn't have to shave that yak
bitsofinfo
@bitsofinfo
Jun 30 2016 03:25
yak hair grows too quickly
Dave Syer
@dsyer
Jun 30 2016 03:25
I don't know where to lay the blame
Talking of which, for that whole time we've been chatting (nearly 30min) I've been waiting for a yak to download.
Stupid hotel wifi.
Stephen Gream
@ThreeToes
Jun 30 2016 03:27
Just use emacs tramp mode and print statements. Problem solved
Dave Syer
@dsyer
Jun 30 2016 03:27
tramp rocks
Stephen Gream
@ThreeToes
Jun 30 2016 03:27
It's how I did all my HPC assignments in uni
Dave Syer
@dsyer
Jun 30 2016 03:28
Print statements are pretty much a last resort for me now (thankfully)
I remember gdb-mode
Stephen Gream
@ThreeToes
Jun 30 2016 03:28
Now I'm kind of ashamed to admit I use IDEs almost exclusively, I've gotten too comfortable
Dave Syer
@dsyer
Jun 30 2016 03:28
Emacs is an IDE. Just a bit primitive.
Stephen Gream
@ThreeToes
Jun 30 2016 03:28
It's a great OS
Dave Syer
@dsyer
Jun 30 2016 03:28
Don't be ashamed of that
Stephen Gream
@ThreeToes
Jun 30 2016 03:28
Too bad it doesn't have a decent text editor
Dave Syer
@dsyer
Jun 30 2016 03:29
LOLZ
I held out against IDEs for a long time.
Stephen Gream
@ThreeToes
Jun 30 2016 03:29
I'm too young to remember a time without them, really
Dave Syer
@dsyer
Jun 30 2016 03:30
Clung to emacs until about 2008 (when Eclipse was already bloatware)
Stephen Gream
@ThreeToes
Jun 30 2016 03:30
I had about 6 months where I only used a terminal, but that's about it.
And that was by choice, not because there wasn't anything else. At some point you have to admit to yourself that GUIs do come in handy sometimes
Stephen Gream
@ThreeToes
Jun 30 2016 03:33
But used Dr Java and eclipse when I started in 2007, moved on to NetBeans when I landed the campus ambassador job with Sun, then switched to IntelliJ last year
I don't regret leaving eclipse
Dave Syer
@dsyer
Jun 30 2016 03:37
I can't work up the energy to move
My first IDE was Kava.
It was cool because it was free
bitsofinfo
@bitsofinfo
Jun 30 2016 03:38
still on eclipse, always have been. As long as I can refactor, jump around to classes, auto import. good enough
Dave Syer
@dsyer
Jun 30 2016 03:38
Same here
bitsofinfo
@bitsofinfo
Jun 30 2016 03:38
dump over to cmdline for anything else
Dave Syer
@dsyer
Jun 30 2016 03:38
Too much muscle memory. Not enough reason to change.
We've seen quite a sizeable migration to IntelliJ in the last 2 years I think
bitsofinfo
@bitsofinfo
Jun 30 2016 03:39
all the other auto-magical stuff... i don't care about.
except when it comes to my microservice foundational frameworks....
:)
Dave Syer
@dsyer
Jun 30 2016 03:40
Spring Boot doesn't need magical IDEs
that's one of its strengths
Stephen Gream
@ThreeToes
Jun 30 2016 03:40
I use IntelliJ because I hate the Eclipse dance I always have to go through to get it set up, and Oracle has just left NetBeans to rot on the vine
bitsofinfo
@bitsofinfo
Jun 30 2016 03:40
yeah, no it doesnt. always loved the spring framework
Stephen Gream
@ThreeToes
Jun 30 2016 03:41
Wow, ok, still no official Gradle support in NetBeans. That was one of the major reasons I switched, the community plugin was a bit meh and I'm too lazy to contribute myself
Spring was what won me over to Java from C#
C# has the better language, but the Java ecosystem is absolutely amazing
bitsofinfo
@bitsofinfo
Jun 30 2016 03:42
the new "official" gradle plugin is pretty nice
for eclipse
but again, i don't use it other than for easier multiproject imports/building
Dave Syer
@dsyer
Jun 30 2016 03:42
(Still waiting for that download)
bitsofinfo
@bitsofinfo
Jun 30 2016 03:42
then to cmd line
bitsofinfo
@bitsofinfo
Jun 30 2016 03:54
so I am dropping compile 'org.springframework.security.oauth:spring-security-oauth2:2.0.10.RELEASE' into my eureka server project (because it needs to do oauth2 as a client against my config server), that works fine, however just including this dependecy is throwing up a 401 prompt on the eurkea server ui
Dave Syer
@dsyer
Jun 30 2016 03:55
That's because it brings in spring-security transitively
so now your app is secure
You can switch it off
bitsofinfo
@bitsofinfo
Jun 30 2016 03:55
yeah, so i need to throw in a resourceserverconfigurator bean and permit all?
or is there a simpler way
Dave Syer
@dsyer
Jun 30 2016 03:55
Depends
I don't think the 401 is an oauth one by default
Just HTTP basic
You can switch it off with a couple of properties
bitsofinfo
@bitsofinfo
Jun 30 2016 03:56
y, so just the basic spring-sec stuff. I'll look there.
Dave Syer
@dsyer
Jun 30 2016 03:56
More spring boot
security.basic.enabled=false and management.security.enabled=false
It'll be in the Boot user guide
bitsofinfo
@bitsofinfo
Jun 30 2016 03:58
yep just found it, thanks again man. works perfect.... and... thats why my 2 nodes were not seeing eachother :)
Włodzimierz Rożkow
@rozhok
Jun 30 2016 11:10
What's the best way to cuztomize ZuulExceptions?
I always get an "ZuulException: GENERAL Error" without any explanation about whats going on
there is some bits of info down in the stacktrace but I have to look into logs instead of getting proper message in http response
i'm not sure that regular @ControllerAdvice will work here
here's the block I need to customize:
catch (HystrixRuntimeException ex) {
    info.put("status", "500");
    if (ex.getFallbackException() != null
            && ex.getFallbackException().getCause() != null
            && ex.getFallbackException().getCause() instanceof ClientException) {
        ClientException cause = (ClientException) ex.getFallbackException()
                .getCause();
        throw new ZuulException(cause, "Forwarding error", 500, cause
                .getErrorType().toString());
    }
    throw new ZuulException(ex, "Forwarding error", 500, ex.getFailureType()
            .toString());
}
Dave Syer
@dsyer
Jun 30 2016 12:52
Where is that code?
Włodzimierz Rożkow
@rozhok
Jun 30 2016 15:20
@dsyer org.springframework.cloud.netflix.zuul.filters.route.RibbonRoutingFilter
bitsofinfo
@bitsofinfo
Jun 30 2016 15:21

for this part of the docs: http://cloud.spring.io/spring-cloud-static/spring-cloud.html#_service_discovery_eureka_clients

this statement (i.e. utterly normal Spring Boot app). In this example we use @EnableEurekaClient explicitly, but with only Eureka available you could also use @EnableDiscoveryClient. Configuration is required to locate the Eureka server. Example:

What specifically does but with only Eureka available ... mean exactly? Im not quite understanding when my app should use EnableEurekaClient vs EnableDiscoveryClient, does discovery client also act as a eureka registry peer or something?

bitsofinfo
@bitsofinfo
Jun 30 2016 15:33
Also, correct me if i am wrong, but I would assume that you can't have things setup where the config-server both registers with Eureka for clients to discover it AND have Eureka store its config in the config server at the same time?
Spencer Gibb
@spencergibb
Jun 30 2016 15:35
@bitsofinfo @EnableDiscoveryClient works for eureka, consul & zookeeper. @EnableEurekaClient ‘extends’ that and only works for eureka. I’ve never been a fan of the specific one.
bitsofinfo
@bitsofinfo
Jun 30 2016 15:36
k
doc was just a little unclear
actually looks like it works, just need to always ensure config server is up before eureka. Gives errors trying to register until eureka is up, which is expected
bitsofinfo
@bitsofinfo
Jun 30 2016 15:47

For discovery first bootstrap, does the eureka.instance.metadataMap support encrypted values?

Also, the example is showing setup for basic auth, but what if my config server is secured w/ oauth2? In my config client apps, i am already creating a custom ConfigServicePropertySourceLocator that wraps a oauth2resttemplate for this.... but if I'd like to try discovery of the config server through eureka.... best way to conf this?

i.e. whatever spring.cloud.config.discovery.enabled: true is enabling, i'd have to construct myself in this ConfigServicePropertySourceLocator I already have, but set it up to hit eureka first to get the config-service location?
Dave Syer
@dsyer
Jun 30 2016 15:54
You might want to look at what they do in PCF (Cloud Foundry)
They use OAuth2 for everything
I don't think it's quite the same approach
bitsofinfo
@bitsofinfo
Jun 30 2016 15:55
Such as alter this and fill in what I have commented with
  @Bean
    @Primary
    public ConfigServicePropertySourceLocator configServicePropertySourceLocator() {
        ConfigClientProperties clientProperties = configClientProperties();

       // !!!!!!!! Use EurekaClient here manually to get config-server URI
     // alter clientProperties

        ConfigServicePropertySourceLocator configServicePropertySourceLocator = new ConfigServicePropertySourceLocator(clientProperties);
        configServicePropertySourceLocator.setRestTemplate(oauth2RestTemplate(clientProperties));
        return configServicePropertySourceLocator;
    }
Dave Syer
@dsyer
Jun 30 2016 15:57
That works.
bitsofinfo
@bitsofinfo
Jun 30 2016 15:57
going to give it a shot
Dave Syer
@dsyer
Jun 30 2016 15:58
In PCF they just autowire the existing ConfigServicePropertySourceLocator and change its rest template
Marcos Barbero
@marcosbarbero
Jun 30 2016 15:59
bitsofinfo
@bitsofinfo
Jun 30 2016 16:01
thx
@marcosbarbero why the dependency on that lombok project and the @Data
Marcos Barbero
@marcosbarbero
Jun 30 2016 16:07
The lombok project generates some source code for you, in my case it will generate getters and setters for this class with @Data: https://github.com/marcosbarbero/spring-cloud-config-client-oauth2/blob/master/src/main/java/com/barbero/cloud/config/oauth2/ConfigClientOAuth2ResourceDetails.java
Maybe I’ve to update my README.md explaining this
The project lombok can be found at this link https://projectlombok.org/
bitsofinfo
@bitsofinfo
Jun 30 2016 17:31
@dsyer definitely works, however if I have this in my bootstrap config class as described above, what is it that triggers the creation of this DiscoveryClient i need below? if i just annotate my config class with @EnableDiscoveryClient no matching bean is available. The only way it shows up is if i turn on spring.cloud.config.discovery.enabled: true which... I am doing myself in code above in my own bootstrap code so I really don't want that. The side affect of spring.cloud.config.discovery.enabled: true is that it also reaches out to eureka (in addition to my code) looking for configserver that does not exist (as my config server's app name is diff)
    @Autowired
    private DiscoveryClient client;
hmm maybe I need this in my spring.factories org.springframework.cloud.client.discovery.EnableDiscoveryClient=\ 11 org.springframework.cloud.netflix.eureka.EurekaDiscoveryClientConfiguration
i mean... this org.springframework.cloud.netflix.eureka.EurekaDiscoveryClientConfiguration
Dave Syer
@dsyer
Jun 30 2016 17:40
I'm not sure I follow.
You want to @EnableDiscoveryClient but you don't want to "reach out to eureka"?
(in any case you definitely don't want to be messing with spring.factories discovery client declarations)
bitsofinfo
@bitsofinfo
Jun 30 2016 17:42

k sorry. So looking at this code: https://gitter.im/spring-cloud/spring-cloud?at=577540d9a0c12d110fd4e2c1

I just need a DiscoveryClient injected into my custom boostrap config code, that is pre-wired to talk to what I have confgured in boostrap.yml (but it does not go off and attempt auto discovery itself as I am doing that) i just want a configured client injected I can use

eureka:
  client:
    serviceUrl:
      defaultZone: http://localhost:8761/eureka/
What I am experiencing is that the only way a matching bean is found for @Autowired private DiscoveryClient client; is if I also have spring.cloud.config.discovery.enabled: true (which I don't need)
Dave Syer
@dsyer
Jun 30 2016 17:45
You need a discovery client in bootstrap, but you don't want the one that comes from spring.cloud.config.discovery.enabled: true?
It seems like you are swimming against the flow
Maybe we should take stock of your actual business requirement
instead of talking about the implementation and getting stuck in the weeds
bitsofinfo
@bitsofinfo
Jun 30 2016 17:47
when my app comes up, I need it to discover the config-server that is registered w/ eureka. Once it gets that connection info, I need it to auth against the config server w/ oauth2
Marcos Barbero
@marcosbarbero
Jun 30 2016 17:52
@dsyer I’ve a question about spring-cloud-starter-stream-rabbit
I’m trying to configure two rabbitmq binders, it works fine
But it is always autoconfiguring a binder looking for a localhost instance
As I’ve debbuged it is happening on the class RabbitAutoConfiguration
Am I missing something? How can I avoid that?
Dave Syer
@dsyer
Jun 30 2016 17:56
spring.rabbitmq.host? (for example)
@bitsofinfo So the only thing that doesn't work for you with Spring Cloud config is the oauth2 authentication with the config server?
Why are we talking about discovery?
bitsofinfo
@bitsofinfo
Jun 30 2016 18:05

@dsyer i just simplified everything and got it working by just doing what you pointed out CF does at https://gitter.im/spring-cloud/spring-cloud?at=57754188632b75030fbd219b

What I was talking about above was based around a solution similar to this which was just too much work and hard to describe as I attempted above: spring-cloud/spring-cloud-config#177

Marcos Barbero
@marcosbarbero
Jun 30 2016 18:06
@dsyer yes… it’s binding spring.rabbitmq.host=localhost by default
Dave Syer
@dsyer
Jun 30 2016 18:14
@marcosbarbero so can't you change the value?
Marcos Barbero
@marcosbarbero
Jun 30 2016 18:15
I’ve added two binders of remote RabbitMQs, it is working fine. The problem is a 3rd binding coming from nowhere with localhost configuration and I don’t know how to disable this localhost binder once it’s not configured at all
The main problem happening is the health status DOWN to the Eureka Server, once the localhost rabbitmq cannot be reached.
Marius Bogoevici
@mbogoevici
Jun 30 2016 18:45
@marcosbarbero you’d probably need to exclude RabbitAutoConfiguration in Boot
Marcos Barbero
@marcosbarbero
Jun 30 2016 18:45
@mbogoevici I just did hehehe it worked
Marius Bogoevici
@mbogoevici
Jun 30 2016 18:46
since Rabbit is on the classpath, it will create a ConnectionFactory
Dave Syer
@dsyer
Jun 30 2016 18:46
Might be useful to add that to the user guide
Marius Bogoevici
@mbogoevici
Jun 30 2016 18:46
cool :)
It definitely would
Marcos Barbero
@marcosbarbero
Jun 30 2016 18:46
yes hehehe Just found it out debuging the source…
@SpringBootApplication(exclude = RabbitAutoConfiguration.class) it did the trick /win
Marius Bogoevici
@mbogoevici
Jun 30 2016 18:47
coolio
I’ll open an issue
thanks for the feedback!
Marcos Barbero
@marcosbarbero
Jun 30 2016 18:50
NP.. Keep the good job and thank you all for this amazing stuff
bitsofinfo
@bitsofinfo
Jun 30 2016 21:22

I have my eureka server setup w/ spring sec basic auth. When i hit localhost:8761/[eureka] in a browser i get a 401 prompt and I login fine w/ the credentials noted below.

Per the docs I configured everything else that will connect as a client with eureka.client.serviceUrl.defaultZone=http://admin:123@localhost:8761/eureka/ (including other peer eureka servers)

Eureka server logs state ....PeerEurekaNodes : Adding new peer nodes [http://admin:123@localhost:8761/eureka/]

However when anything tries to register w/ this config I see: .....RetryableEurekaHttpClient : Request execution failure with status code 403; retrying on another server if available

Dave Syer
@dsyer
Jun 30 2016 22:11
I guess you need to look at the logs on the server (DEBUG logs will tell you why it sent a 403).
Maybe you have some custom security config on the eureka server. You forgot to switch off CSRF protection?
bitsofinfo
@bitsofinfo
Jun 30 2016 23:56
yep .csrf().disable(). did the trick