These are chat archives for spring-cloud/spring-cloud

21st
Oct 2016
Stian Lågstad
@stianlagstad
Oct 21 2016 08:00

Is this sufficient to cache responses from (hystrix)feign client methods when I have many threads?

@FeignClient(
        name = "myclient",
        url = "${endpoint.myService.url}",
        configuration = MyClientConfiguration.class)
public interface MyClient {

    @RequestMapping(
            method = RequestMethod.GET,
            value = "{group}/members")
    @Cacheable(value = "Members")
    List<Stop> getMembers(
            @PathVariable("groups") String group);
}

I have the cache Membersin a SimpleCacheManager that I create in the configuration class for the (hystrix)feign client:

@Configuration
@EnableCaching
public class MyClientConfiguration {
    @Bean
    public CacheManager cacheManager() {
        SimpleCacheManager simpleCacheManager = new SimpleCacheManager();

        GuavaCache members = new GuavaCache(
                "Members",
                CacheBuilder.newBuilder()
                .expireAfterAccess(1, TimeUnit.DAYS)
                .build());

        simpleCacheManager.setCaches(Arrays.asList(
                members));

        return simpleCacheManager;
    }

}

I suspect that this is not working: It looks like the call to MyClient.getMembers() is actually being carried out a lot. Any advice?

Dave Syer
@dsyer
Oct 21 2016 08:03
I don't know how @Cacheable works in much detail, but I dare say the fact that your bean is defined as an interface makes it complicated. Maybe ask @snicoll in the #spring-boot room?
Does it work with a regular @Bean?
If it does then I would ask for more help
Stéphane Nicoll
@snicoll
Oct 21 2016 08:03
Howdy
If you’re using boot, you don’t need to do all that.
There’s a regression with cache annotation detections on interfaces with 4.3, we’re working on it —> https://jira.spring.io/browse/SPR-14781
I’ve seen a few Feign users complaining about that here and there. Looks like you’re affected as well
@stianlagstad unrelated by that whole config file can be resumed by @EnableCaching only and spring.cache.guava.spec=expireAfterAccess=1d
Stian Lågstad
@stianlagstad
Oct 21 2016 08:09
Thank you both! Do you mean that I could remove the CacheManager bean and the @Cacheable annotation, @snicoll?
Stéphane Nicoll
@snicoll
Oct 21 2016 08:10
The CacheManager bean config is equivalent to that code above. You need the marker to cache things of course :)
Spring Boot ain’t magic :)
so, keep @Cacheable
Also it means that any cache in the system is created with an expiration of 1 day. In your current setup, this is what you want but if you deviate from that you need to customize things a bit more
Stian Lågstad
@stianlagstad
Oct 21 2016 08:11
Spring [everything] is still quite magic to me :)
Stéphane Nicoll
@snicoll
Oct 21 2016 08:11
you can also expose individual caches as beans and we’ll find them in the context for you. Check the spring boot doc
Stian Lågstad
@stianlagstad
Oct 21 2016 08:12
keep @Cacheable, remove the CacheManager bean?
Stéphane Nicoll
@snicoll
Oct 21 2016 08:12
Yes
ok so basically you don’t need to configure the CacheManager bean. Spring Boot configures things for you if you don’t have any opinion
Just adding guava means we’ll find out for you and create a default CacheManager for you. If you add the property above, then you’re tuning how Spring Boot creates the cache.
Does that make sense?
Stian Lågstad
@stianlagstad
Oct 21 2016 08:13
I understand. I thought I needed to have a CacheManager setup, and a cache within that for each (hystrix)feign client method
Stéphane Nicoll
@snicoll
Oct 21 2016 08:14
Have you had a look to the doc?
Stian Lågstad
@stianlagstad
Oct 21 2016 08:14
Yes. When I created the cache I thought I wanted different expiry dates for all the different caches
But you're also saying that caching does not work atm, in ref to https://jira.spring.io/browse/SPR-14781 ?
Stéphane Nicoll
@snicoll
Oct 21 2016 08:15
It’s not working because the annotation is on an interface. I am not sure it’s broken for all use cases. I said you might be affected
Looks like you are, there’s nothing wrong in your config there.
Stian Lågstad
@stianlagstad
Oct 21 2016 08:19
Thanks a lot :D I will be watching that issue
The related question I had originally: Do I need to do something special to make the caching work when the hystrix threadpool is set to, let's say, 100? I guess I should use set @Cacheable(value = "Members", sync=true)?
Stéphane Nicoll
@snicoll
Oct 21 2016 08:23
I wouldn’t use that
especially with an expiration of 1d - I mean you could have concurrent users asking the same thing and not rely on the cache. That’s no problem IMO compared to the synchronization you’ll induce for every calls
Stian Lågstad
@stianlagstad
Oct 21 2016 08:25
k - thanks!
I'm getting java.lang.IllegalStateException: No CacheResolver specified, and no bean of type CacheManager found. Register a CacheManager bean or remove the @EnableCaching annotation from your configuration. after I removed the CacheManager bean, so I guess I am affected?
Stéphane Nicoll
@snicoll
Oct 21 2016 08:28
affected by what? It’s a completely different error
Are you using Spring Boot for a start? I would assume you do but it’s hard to tell with a few lines of code
Stian Lågstad
@stianlagstad
Oct 21 2016 08:30
Using spring boot. These are my spring dependencies:
    // Spring
    compile group: 'org.springframework.cloud', name: 'spring-cloud-starter-feign'
    compile group: 'org.springframework.boot', name: 'spring-boot-starter-web'
    compile group: 'org.springframework.boot', name: 'spring-boot-starter-actuator'
    compile group: 'org.springframework', name: 'spring-context-support'
    compile group: 'org.springframework.cloud', name: 'spring-cloud-starter-hystrix'
    compile group: 'org.springframework.cloud', name: 'spring-cloud-starter-hystrix-dashboard'
    compile group: 'org.springframework.cloud', name: 'spring-cloud-starter-sleuth'
Stéphane Nicoll
@snicoll
Oct 21 2016 08:30
Run with debug mode on and check why CacheAutoConfiguration didn’t match
replace 4th dependency by spring-boot-starter-cache (that won’t fix the error, just better on the eye)
Stian Lågstad
@stianlagstad
Oct 21 2016 08:32
If I remember correctly I needed context-support for testing
Stian Lågstad
@stianlagstad
Oct 21 2016 08:40

I've tried a few different things to enable debug mode (taken from here http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-logging.html):

  • debug: true in bootstrap.yml
  • added --debug as a program argument in my intellij run config
  • set log level to DEBUG in bootstrap.yml

But I'm still getting this: Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled. How exactly do I enable debug mode?

Stéphane Nicoll
@snicoll
Oct 21 2016 08:41
We’re running out of options there I am afraid. The two first should definitely do that.
There’s “Enable debug output” in your run config for one. It’s a checkbox
If you don’t have that, try -Ddebug in VM Options of your run config
Stian Lågstad
@stianlagstad
Oct 21 2016 08:44
Thanks. That checkbox didn't affect anything. Adding -Ddebug in VM Options didn't change anything either. Still getting Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
Stéphane Nicoll
@snicoll
Oct 21 2016 08:44
that does not make any sense, sorry
Do you have debug logs ?
(that you didn’t have before?)
Can you share the project? Because I’ve been guessing for a while now...
Stian Lågstad
@stianlagstad
Oct 21 2016 08:47
Thank you for the time and suggestions! :) I do not have any debug logs. I'm gonna double- and triplecheck everything, and if nothing else works I'll create an example project in an attempt to reproduce the error
Stéphane Nicoll
@snicoll
Oct 21 2016 08:48
You’re not starting the thing you think you start. Or there’s something else that is happening you’re not telling. I am happy to help you but I need to be able to put my hands on it. Sorry.
Stian Lågstad
@stianlagstad
Oct 21 2016 08:48
I understand that - thanks!
Stian Lågstad
@stianlagstad
Oct 21 2016 09:08

I finally got debug logs after doing invalidate cache+restart in intellij. However, I still get Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled. when I remove my CacheManager beans. If I don't remove my CacheManager beans, and check my /env endpoint, I can see that debug is indeed set to true:

  "configService:file://home/[removed]/configuration-repo/[removed]/application.yml": {
    "health.config.enabled": false,
    "logging.level.[removed]": "DEBUG",
    "debug": true,
...

If this still makes zero sense then I will attempt to reproduce the issue in a new project so that I can show you.

Stéphane Nicoll
@snicoll
Oct 21 2016 09:09
What version of Spring Boot is this?
Stian Lågstad
@stianlagstad
Oct 21 2016 09:09
1.4.1.RELEASE
Stéphane Nicoll
@snicoll
Oct 21 2016 09:09
doesn’t make any sense, sorry
Stian Lågstad
@stianlagstad
Oct 21 2016 09:09
Alright - thanks. I'll create a new project and try to reproduce it
Stéphane Nicoll
@snicoll
Oct 21 2016 09:11
Please share your setup otherwise it’s a bit useless
Stian Lågstad
@stianlagstad
Oct 21 2016 11:17
I've been unable to reproduce the error. In my attempt everything seems to work well: https://github.com/stianlagstad/spring-boot-hystrix-caching-demo . There must be something wrong with the configuration of my main project
Sergey Serdyuk
@neee
Oct 21 2016 12:11
Hello everbody, I have some zuul filter which adds headers but it need also parse some request params and adds them to body (urlencoded). I made a wrapper where I rewrite some fields (AlseServlet30RequestWrapper(HttpServletRequest request, String method, byte[] contentData, String contentType) {
super(request)
this.method = method
this.request = request
this.contentType = contentType
setContentData(contentData)
}) but when I make request my body is empty as was before. Maybe somebody resove the same problem?
Dave Syer
@dsyer
Oct 21 2016 12:22
Don't know. The servlet API doesn't like you to manipulate the body if it is a form though.
It wants you to use the request params and parts.
Probably better to go with the flow on that?
Stéphane Nicoll
@snicoll
Oct 21 2016 12:44
@stianlagstad if you’re playing with the Logger in your project, don’t. That’s what might prevent the auto-configuration report to show up. You can also debug the code I’ve referenced earlier.
nmquyet
@nmquyet
Oct 21 2016 13:49
@marcosbarbero
I find out that config server actually resolve some environment placeholder which result in changing the origin content of resource on git repository. Is this an expected behavior?
    // ResourceController.java
    synchronized String resolve(String name, String profile, String label, String path)
            throws IOException {
        if (label != null && label.contains("(_)")) {
            // "(_)" is uncommon in a git branch name, but "/" cannot be matched
            // by Spring MVC
            label = label.replace("(_)", "/");
        }
        StandardEnvironment environment = prepareEnvironment(
                this.environmentRepository.findOne(name, profile, label));

        // ensure InputStream will be closed to prevent file locks on Windows
        try (InputStream is = this.resourceRepository.findOne(name, profile, label, path)
                .getInputStream()) {
            String text = StreamUtils.copyToString(is, Charset.forName("UTF-8"));
            return resolvePlaceholders(environment, text);
        }
    }
Dennis Melzer
@SirWayne
Oct 21 2016 13:56
Hello, how could i change the BusJacksonMessageConverter bean in spring cloud bus? I would like to use an own message converter. Thanks
Sergey Serdyuk
@neee
Oct 21 2016 13:57
@dsyer ye but I need pass GET query to POST between external systems
and the second system reads only body params
Dave Syer
@dsyer
Oct 21 2016 14:10
I don't think it matters where you get the data from does it?
It's just a different API than the one you are using
Dave Syer
@dsyer
Oct 21 2016 14:16
@SirWayne there's a section on message converters in the user guide (http://cloud.spring.io/spring-cloud-static/Camden.SR1/#_customizing_message_conversion)
Dennis Melzer
@SirWayne
Oct 21 2016 14:20

@dsyer Yes i tried it with

spring.cloud.stream.bindngs.springCloudBusInput.content-type=application/octet-stream
spring.cloud.stream.bindngs.springCloudBusOutput.content-type=application/octet-stream

and the bean

public class MyConverter extends AbstractMessageConverter {

    protected MyConverter () {
        super(MimeTypeUtils.APPLICATION_OCTET_STREAM);
    }

    @Override
    protected boolean supports(final Class<?> aClass) {
        return RemoteApplicationEvent.class.isAssignableFrom(aClass);
    }
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:29
the documentation reference is wrong
we don’t support the AbstractMessageConverter anymore
actually, we do, sorry - I was thinking of the AbstractFromMessageConverter
spring.cloud.stream.bindngs.springCloudBusInput.content-type=application/octet-stream (note that bindngs is misspelled)
Dennis Melzer
@SirWayne
Oct 21 2016 14:32
Ahhh there is a typo in the documentation
spring.cloud.stream.bindngs
@mbogoevici thanks found it ;)
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:32
ook, thanks for the feedback
that’s a wrong link
we need to correct that
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:34
yes
thanks for the notice :)
Dennis Melzer
@SirWayne
Oct 21 2016 14:34
ok :smile: +1:
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:35
here’s how it works though - you can register a converter associated to a MimeType - e.g. application/json or application/xml or whatever
the two methods convertFrom/convertTo describe how to convert from/to a transported format - e.g. String or byte[]
Dennis Melzer
@SirWayne
Oct 21 2016 14:36
Yeah but the MessageConverterConfigurer found application/json again for the output
not the octet stream
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:37
well, what exactly are you trying to do?
Dennis Melzer
@SirWayne
Oct 21 2016 14:37
Try to use my custom converter don't want to use json
Something like this instead of json
public class BusProtoStuffMessageConverter extends AbstractMessageConverter {

    protected BusProtoStuffMessageConverter() {
        super(MimeTypeUtils.APPLICATION_OCTET_STREAM);
    }

    @Override
    protected boolean supports(final Class<?> aClass) {
        return RemoteApplicationEvent.class.isAssignableFrom(aClass);
    }

    @Override
    public Object convertFromInternal(final Message<?> message, final Class<?> targetClass,
            final Object conversionHint) {
        final Object payload = message.getPayload();

        if (payload instanceof byte[]) {
            final Schema<Object> schema = (Schema<Object>) RuntimeSchema.getSchema(targetClass);
            final Object deserializeEvent = schema.newMessage();
            ProtobufIOUtil.mergeFrom((byte[]) message.getPayload(), deserializeEvent, schema);
            return deserializeEvent;
        }
        return null;
    }

    @Override
    protected Object convertToInternal(final Object payload, final MessageHeaders headers,
            final Object conversionHint) {
        final Schema<Object> schema = (Schema<Object>) RuntimeSchema.getSchema(payload.getClass());
        final LinkedBuffer buffer = LinkedBuffer.allocate();
        final byte[] protostuff;
        try {
            protostuff = ProtostuffIOUtil.toByteArray(payload, schema, buffer);
        } finally {
            buffer.clear();
        }

        return protostuff;
    }

}
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:39
ok
so in that case you should use a different mimetype
you can say application/binary+protobuf
Dennis Melzer
@SirWayne
Oct 21 2016 14:40
ok i will try that
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:40
then use that on the input/output bindings
then whenever the conversion system finds that on output it will look for your converter
Dennis Melzer
@SirWayne
Oct 21 2016 14:40
ok thanks i will try that :)
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:41
on inbound messages, it will find a content type header with application/binary+protobuf and it will use your registered message converter to decode
we do the same thing for avro
so if you do it, you might want to consider a contribution :D
it’s actually very close to your protobuf use case
Dennis Melzer
@SirWayne
Oct 21 2016 14:45
Thanks :+1: . If it works i can do a contribution (first i have to check the internal company cla's ). It will be first open source here https://github.com/eclipse/hawkbit
Marius Bogoevici
@mbogoevici
Oct 21 2016 14:46
sounds good!
Dennis Melzer
@SirWayne
Oct 21 2016 15:44
@mbogoevici do you know why get the RemoteApplicationEvent instead of my concret subclass in the MessageConverter?
    @Override
    public Object convertFromInternal(final Message<?> message, final Class<?> targetClass,
            final Object conversionHint) {
Marius Bogoevici
@mbogoevici
Oct 21 2016 15:49
not sure - the converter should be aware of subclassing - the original JSON conversion in Bus registers JSON submodules
need to drop now, but if you can push an example I can take a look later (opening an issue should help)
Dennis Melzer
@SirWayne
Oct 21 2016 15:51
Mhm ok, i can set a header to get the orginal class type. I think the jackson message converter set a header too. I will check that on monday ;). Nice weekend
Marius Bogoevici
@mbogoevici
Oct 21 2016 16:08
have a nice weekend
a header would make sense
Marcos Barbero
@marcosbarbero
Oct 21 2016 17:40
@nmquyet I was not aware of that. I'll try to look it forward, anyhow I'm out of computer right now and it may take longer than your needs :/
suren343
@suren343
Oct 21 2016 18:52
Hey..I am using spring-cloud config for first time and have a query..My spring config is as below..will my gateway going to be shut if my registry has been shutdown or still accessible?
cloud:
config:
fail-fast: true
Spencer Gibb
@spencergibb
Oct 21 2016 18:53
Not sure if I understand your question.
suren343
@suren343
Oct 21 2016 18:54
i have a registry and gateway connected to this registry...I have fail-fast=true in my gateway config...Now if is stop my registry, will my gateway is accessible or it shouldn't be accessible?
Spencer Gibb
@spencergibb
Oct 21 2016 18:55
fail-fast means a config client won’t start if the config-server is not available.
suren343
@suren343
Oct 21 2016 18:58
@spencergibb : my gateway config-server points to registry...Currently both registry and gateway are running..now if i stop my registry whether the gateway is accessible or not?
Spencer Gibb
@spencergibb
Oct 21 2016 18:59
fail-fast is for startup
suren343
@suren343
Oct 21 2016 19:01
so it means, we cant start the gateway if registry is not up and running ..it has nothing do with registry status once both are up and running i.e. even if gateway accesible , even its config server is down
Spencer Gibb
@spencergibb
Oct 21 2016 19:02
yes, fail-fast only affects startup
suren343
@suren343
Oct 21 2016 19:02
Thanks Spencer