These are chat archives for spring-cloud/spring-cloud

17th
Aug 2016
Marcos Barbero
@marcosbarbero
Aug 17 2016 00:53
@dsyer thanks
Ali Moghadam
@alighm
Aug 17 2016 01:44
@dsyer @spencergibb, hey guys, simple question. Is there a way I can do form url-encode POST using FeignClient?
Marcos Barbero
@marcosbarbero
Aug 17 2016 01:47
@alighm have you tried something like this?
@RequestMapping(consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
Ali Moghadam
@alighm
Aug 17 2016 01:51
@marcosbarbero what should my parameter look like then?
Map<String, String> param) ?
Marcos Barbero
@marcosbarbero
Aug 17 2016 01:52
it should work
Ali Moghadam
@alighm
Aug 17 2016 01:52
Could not write request: no suitable HttpMessageConverter found for request type [java.util.HashMap] and content type [application/x-www-form-urlencoded]
nope...
Marcos Barbero
@marcosbarbero
Aug 17 2016 01:53
let me find a sample
Ali Moghadam
@alighm
Aug 17 2016 01:53
thank you!!
Marcos Barbero
@marcosbarbero
Aug 17 2016 01:55
the sample that I have is using pure Feign from netflix, not the spring-cloud wrapper
Ali Moghadam
@alighm
Aug 17 2016 01:55
hmmm
Marcos Barbero
@marcosbarbero
Aug 17 2016 01:56
@RequestLine("POST /endpoint")
    @Headers("Content-Type: application/x-www-form-urlencoded")
    @Body(“param1={param1}&param2={param2}")
    AccessToken token(@Param(“param1") String param1, @Param(“param2") String param2);
Ali Moghadam
@alighm
Aug 17 2016 01:56
I got it working!!!
I’m using the following:
This message was deleted
@RequestMapping(method = RequestMethod.POST, value = "/oauth/token",
            consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
    ResponseEntity<AuthResponse> getToken(MultiValueMap<String, String> param);
Marcos Barbero
@marcosbarbero
Aug 17 2016 01:58
Great :D
Ali Moghadam
@alighm
Aug 17 2016 01:59
MultiValueMap<String, String> params = new LinkedMultiValueMap<>();
:-) yeah got it working!! Thanks for the help though @marcosbarbero
Marcos Barbero
@marcosbarbero
Aug 17 2016 01:59
you’re welcome
Jose Armesto
@fiunchinho
Aug 17 2016 06:15
thank you for the answers
@dsyer in my first statement I said that it was a Spring Boot API, which if I'm not mistaken, it uses Spring MVC
I don't know what you mean with provide an example, after I posted the exact code I'm using as example?
@spencergibb so you are saying that neither Spring Boot or Spring Cloud will support returning Observable/Single from the endpoints?
any alternative then?
Dave Syer
@dsyer
Aug 17 2016 06:20
Spencer said the opposite (Spring Cloud has support for returning observables).
but a snippet of code is not a full sample. If you could post a complete project (as minimal as possible) it will be much easier to analyze.
And, just for the sake of clarity, a Spring Boot app doesn't mean you are using Spring MVC (and Spring MVC doesn't support observables out of the box), so it was a reasonable question to ask.
(Even if a sample would have made it redundant)
Jose Armesto
@fiunchinho
Aug 17 2016 06:26
yeah, it will definitly go away with boot 2 and spring 5
I understood that future versions won't support it
Dave Syer
@dsyer
Aug 17 2016 06:29
He meant that spring cloud will drop its support in favour of the native features in spring 5.
Jose Armesto
@fiunchinho
Aug 17 2016 06:30
spring mvc appears as dependency for spring boot, so I guess it was also reasonable to assume that Spring Boot uses Spring MVC, at least for the non-contributor eye heheh
Dave Syer
@dsyer
Aug 17 2016 06:30
Sorry if that wasn't clear
Spring Boot has no dependency on MVC at its core. It's optional.
Some people use jersey for instance
Or CXF
Or Ratpack
Jose Armesto
@fiunchinho
Aug 17 2016 06:31
I'm yet to find one, I guess
I'll try to upload the same code I pasted here in an github repository as an example then
Dave Syer
@dsyer
Aug 17 2016 06:32
Thanks
Jose Armesto
@fiunchinho
Aug 17 2016 06:32
thank you!
Ali Moghadam
@alighm
Aug 17 2016 06:33
so I have a very quick question regarding Feign. I have a custom configuration outside of my componentscan (SpringBootApplication scan) and in my FeignClient I am using the configuration to access the custom config:
@FeignClient(name = “foo", url = "http://127.0.0.1:9090", configuration = { AuthFeignConfig.class })
however my AuthFeignConfig.class is not being loaded...
any suggestions?
Dave Syer
@dsyer
Aug 17 2016 06:36
Did you actually use the client yet (it's config is lazy loaded on first use)?
Ali Moghadam
@alighm
Aug 17 2016 06:36
yes
Dave Syer
@dsyer
Aug 17 2016 06:36
s/it's/its/
Do you have 2 feign clients with the same name?
Ali Moghadam
@alighm
Aug 17 2016 06:38
hmmm
yes
lol
thats it?
Dave Syer
@dsyer
Aug 17 2016 06:38
Probably
There's a gh issue open to support multiple clients to the same service
But it's not supported yet
Ali Moghadam
@alighm
Aug 17 2016 06:38
lemme test it with a different name
still
so if I put the @Configuration inside package scan, it works fine but however that would apply to all other FeignClients I assume
and that is not what I want
Dave Syer
@dsyer
Aug 17 2016 06:42
Yes
Ali Moghadam
@alighm
Aug 17 2016 06:42
so I have it outside of ComponentScan, and have it referenced with configuration from the FeignClient
I hit the Client and it doesn’t load the Configuration
Dave Syer
@dsyer
Aug 17 2016 06:42
Works for me
You might have to make a sample app to illustrate the issue
Ali Moghadam
@alighm
Aug 17 2016 06:43
true
thank you Dr Syer!
found the issue!!!
not only should I change the class name but the actual feign name had to be changed!
Dave Syer
@dsyer
Aug 17 2016 06:47
Yes, indeed
Ali Moghadam
@alighm
Aug 17 2016 06:48
I had changed the class name and not the name, even though I was using url
Dave Syer
@dsyer
Aug 17 2016 06:48
The name is a key in a map of child contexts
Ali Moghadam
@alighm
Aug 17 2016 06:48
its working
ahhhh
got it!!!
Dave Syer
@dsyer
Aug 17 2016 06:48
Good
Ali Moghadam
@alighm
Aug 17 2016 06:48
much love Dr Syer, Thank You!!!!
we never stop learning from you :smile:
Johan van den Berg
@johanvdb
Aug 17 2016 08:35
Does spring-cloud-stream support the request-reply concept as described here? http://docs.spring.io/spring-amqp/reference/htmlsingle/#request-reply
Johan van den Berg
@johanvdb
Aug 17 2016 11:21
So. In the spring cloud microservices environment, I have a db service that needs to check if a db entry exists, and then create it if it doesn't. I have many requests using the same entry. If the requests come in at the same time, I will get a constraint violation because many requests will try to create the entry. In the monolithic application scenario, I used to use thread synchronisation to ensure that only one tries to create at a time. With microservices and the potential that there will be many instances of my app running, I need some form of shared locking. Initially I decided to just let the other inserts fail and use spring-retry to try getting the shared db entry again (since clearly it exists if I get a constraint violation), but hibernate isn't too happy to keep on using the session when there has been an exception on it. I cannot allow the transaction to fail / rollback, so I need another way of locking across the cluster to ensure that only one node does the inserts while the others wait for it to insert. I really hope others have had similar issues and have the best pattern in the scenario.
Daniel Lavoie
@daniellavoie
Aug 17 2016 11:24
I won't be of much help but I have to get it out. Spring Cloud is about scaling. SQL transaction doesn't scale. Atomicity is part of its reason to be.
A database supporting distributed locks (like redis) would be an answer I suppose.
Dave Syer
@dsyer
Aug 17 2016 11:29
It's OK to use a relational database to manage a lock. But it doesn't scale indefinitely.
Neither does redis
Redis is probably worse for locks
Daniel Lavoie
@daniellavoie
Aug 17 2016 11:30
Good to know :)
Dave Syer
@dsyer
Aug 17 2016 11:31
I don't think it has a HA mode that supports locks for instance
I suspect @johanvdb needs to find a way to roll with the punches (roll back the transaction and start again if the lock can't be taken).
Johan van den Berg
@johanvdb
Aug 17 2016 11:32
I was hoping that one day when my service is used extensively, I would both scale the java service using spring-cloud as well as the db using vitess (http://vitess.io/). I really don't mind the insert failing, my code can just say "ah ok, lets try to get the one that is clearly already there" but jpa is making it pretty difficult to do so. I have to basically accept that any service call might fail and I would just need to try it again until it succeeds from a client point of view.
Yeah, thats what I was hoping for @dsyer, I just need to find a way of doing so without affecting the caller
Let me try refactor to have the service layer start / rollback the transaction and let the controller recall the service layer if it fails. Feels horrible but might be the best for now
Daniel Lavoie
@daniellavoie
Aug 17 2016 11:37
I'm interested in understanding your use case. How come do you end up with multiple requests trying to modify something that might not exists yet ?
Johan van den Berg
@johanvdb
Aug 17 2016 11:38
Its an accounting service that services millions of customers. Basically I need to say "if the customer's account doesn't exist yet, create it with a 0 balance before trying to adjust it".
There are many aspects as well where internally, I don't create periodic entries if there was no activity on an account for a period, so the periodicals need to be created "on the fly" if activity happens.
Daniel Lavoie
@daniellavoie
Aug 17 2016 11:43
The Dr speaks truth once again :smile: handling the exception feels like something I would do.
Marcos Barbero
@marcosbarbero
Aug 17 2016 11:55
Maybe I'm out of the context but looks like an immutable approach would work better than a transactional SQL schema.
Johan van den Berg
@johanvdb
Aug 17 2016 11:57
Excuse the ignorance @marcosbarbero but could you perhaps elaborate?
Marcos Barbero
@marcosbarbero
Aug 17 2016 11:59
Let’s talk more about the scenario. I’m just guessing right here.. tell me if I’m wrong
you have to manage an account balance from a customer where it may have a balance and it will have to handle a lot of concurrency on it all the time
Daniel Lavoie
@daniellavoie
Aug 17 2016 12:00
If the data you try to insert has an identifier that is not generated, you could use a persistence system that support INSERT OR UPDATE operation. Hence you would end up inserting the same data without worrying about constraint exception.
Marcos Barbero
@marcosbarbero
Aug 17 2016 12:00
is that your scenario?
Daniel Lavoie
@daniellavoie
Aug 17 2016 12:01
I suposed this is what you have in mind Marcos ?
Marcos Barbero
@marcosbarbero
Aug 17 2016 12:02
What I’ve in mind if to runaway from a SQL schema and try to use immutability
something like http://www.datomic.com/
but it depends a lot on the problem we’re facing here. If you can share a bit more it will help
Johan van den Berg
@johanvdb
Aug 17 2016 12:06

Take for example the following scenario:

I have a statistics microservice, that has to increment a counter for every "hit" per day. So there is a record in the db for every day, with an id and a count. The microservice is called from many different places, for many different "hits" that all need to just increment this counter for this day. However, I don't want to pre-populate the DB with counters for the next 20 years, and I definitely don't want counters for days that have no hits, so the logic is to create the counter in the DB the first time a "hit" comes in for the day. In reality, I have many many different counters for the day, and they will be hammered concurrently, so there will be cases where "check if counter exists, and if not create it" will run into race condition.

I also cannot afford to mis a single "hit", since the counter will be used for financial purposes.
Marcos Barbero
@marcosbarbero
Aug 17 2016 12:10
I see.
I feel your ping if you cannot afford to miss a single hit hehehe
Daniel Lavoie
@daniellavoie
Aug 17 2016 12:11
Distributed counters are an interesting topic which goes beyond Spring Cloud I guess. Cassandra has some but in my memory they aren't guaranteed.
Marcos Barbero
@marcosbarbero
Aug 17 2016 12:11
If you could I would suggest you to use InfluxData and UDP for that, but it wouldn't have any guarantee to count every hit
Johan van den Berg
@johanvdb
Aug 17 2016 12:22
Thanks everyone for the very valuable input. I'm going to try for now a scenario where I let the transaction roll back and retry outside of the transaction, to see if JPA plays along. If it doesn't I go back to the drawing board and try some of the other approaches.
Daniel Lavoie
@daniellavoie
Aug 17 2016 12:23
Good luck, your use case requires some good studies on the subjects of distributed databases. None is perfect.
Marcos Barbero
@marcosbarbero
Aug 17 2016 12:32
Good luck @johanvdb if I figure out any other approach I drop you a message
Johan van den Berg
@johanvdb
Aug 17 2016 12:35
Thanks! So far its looking very promising. I added @Retryable on the controller method, @Transaction on the service layer, and now the whole tran rolls back if the race happens, and then the whole tran is retried again. I am hitting it with 20 concurrent threads in JMeter and my issues seems to have disappeared. I log the exceptions, so I can see them happening in the background, but JMeter is non the wiser and the totals for all counters so far seem healthy.
Marcos Barbero
@marcosbarbero
Aug 17 2016 12:37
:clap:
Try something out with http://gatling.io/#/ to make your concurrent tests
Johan van den Berg
@johanvdb
Aug 17 2016 12:39
Ah very nice, thanks, will definitely play around with it!
Heiko Scherrer
@openwms
Aug 17 2016 16:10
design question: Seems to me using a loadBalanced RestTemplate is an alternative (w/ higher abstraction) to Feign, isn't it? Feign couples consumer and provider more than the approach with @LoadBalanced (Ribbon, Service Discovery) does. A caller (consumer smells more like SOA) has more flexibility in parsing (and handling) the response with a loadBalanced RestTemplate than with Feign, right? I did my experience with Feign and tend to replace Feign by a loadBalanced RestTemplate instead.
Spencer Gibb
@spencergibb
Aug 17 2016 16:12
They are different styles, feign is declarative where rest template is imperative. Feign has some very specific constraints you need to work in.
regarding ribbon, the work similarly.
you can share a feign interface, where you can’t with rest template
Dave Syer
@dsyer
Aug 17 2016 16:16
I think the Feign approach is more coupled to DTO types than RestTemplate is intrinsically, which might make it more brittle.
But most people will use RestTemplate with POJOs/DTOs as well
So I think both approaches are open to the same threats
To be a tolerant reader you have to pay special attention in the design and unmarshalling of the POJOs
Spring and Jackson make that quite flexible (for instance)
JAXB not so much
Stephan L
@stephlag
Aug 17 2016 16:24
Hi @spencergibb I'm working on changing PR #1221
Spencer Gibb
@spencergibb
Aug 17 2016 16:24
ok
Stephan L
@stephlag
Aug 17 2016 16:25
Is it a compilation error or at runtime?
Spencer Gibb
@spencergibb
Aug 17 2016 16:26
IRC it’s a compilation error because the constructor used in sleuth is gone and it needs to be compatible with the older version of sleuth as well. So we can’t remove the existing constructors.
Stephan L
@stephlag
Aug 17 2016 16:27
which sleuth module?
Spencer Gibb
@spencergibb
Aug 17 2016 16:30
actually it is runtime
Stephan L
@stephlag
Aug 17 2016 16:31
OK I understand why I did not see the issue as this is an older version of Spring cloud sleuth than the one I have
Spencer Gibb
@spencergibb
Aug 17 2016 16:31
yeah, needs to be backwards compatible
Dave Syer
@dsyer
Aug 17 2016 16:32
Or fowards comatible really
Stephan L
@stephlag
Aug 17 2016 16:33
what do you suggest? I was thinking of calling new ZuulProperties() in the old constructor
Spencer Gibb
@spencergibb
Aug 17 2016 16:37
yes, you could make it non-final with a setter as well
and don’t use lombok since we’re phasing it out.
Stephan L
@stephlag
Aug 17 2016 16:37
ok
Heiko Scherrer
@openwms
Aug 17 2016 16:47
@spencergibb @dsyer Thanks for your reply. Both answers helped me to go further in decision taking. One of my major points using the RestTemplateover Feign is the ability to do proper response handling (especially in case of errors). My experience with Feign so far: Great abstraction with limitations. In a former project with three microservices only, I've used Feign in an anti-pattern way - like it is described in the SpringCloud documentation: Having an Java interface in between consumer and provider. At some point I needed to inject the request, response or headers object on provider side, that limits the interface description. Another limitation was the exception translation done by Feign. But anyway, I think both technics have their place in a distributed system. I'll try to prefer RestTemplateand will consider Feign whenever it gets boring to deal with huge standard interfaces. The aspect of DTO remains for both solutions. With Feign I have two different representations on both sides (p & c) that need to be somehow similar to be mapped by the JsonDeserializer. With the RestTemplate the consumer has also a representation that needs to be mapped. Will try to compare both. Thanks for your fast reply!
bitsofinfo
@bitsofinfo
Aug 17 2016 18:47

I have a feign interface with a return type like this public ResponseEntity<MyObject> getX();

I need to have a custom unmarshaller for MyObject, I'd like to just piggyback my unmarshaller onto whatever SpringDecoder is using (jackson via an HttpMessageConverter?) because I like that it deals with ResponseEntity already nicely.

How can I get my own unmarshaller hooked into this?

Spencer Gibb
@spencergibb
Aug 17 2016 18:52
We use spring boot’s HttpMessageConverters
it’s a bean
bitsofinfo
@bitsofinfo
Aug 17 2016 18:53
cool, yeah just tracking that down now, want to see how its created so I can add my own
Spencer Gibb
@spencergibb
Aug 17 2016 18:54
new HttpMessageConverters(true, <yourconverter>). the boolean is addDefaultConverters.
bitsofinfo
@bitsofinfo
Aug 17 2016 18:55
oh, so "true" will append to springs defaults?
Spencer Gibb
@spencergibb
Aug 17 2016 18:55
that’s what the javadoc’s say
bitsofinfo
@bitsofinfo
Aug 17 2016 18:55
great, thanks for the quick pointer!
or this way I guess too WebMvcConfigurerAdapter and @Override configureMessageConverters()
Spencer Gibb
@spencergibb
Aug 17 2016 19:00
yup