Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Danilo Gomes
    Hey, guys, how are you? Can I ask you something about spring cloud aws messaging on this room?
    Am I in the right place? :D
    Danilo Gomes

    I want know if I can control the number of consumers when using Spring Cloud AWS Messaging and @MessageMapping annotation.
    If I understood, I should create my own AsyncTaskExecutor. I really don't know if this is the right way.

    Let me know if you could help me, guys. Thanks in advance.

    Kyle Anderson
    @danilocgomes_twitter I've got a similar question. The default polling settings for SQS by way of the @SqsListener annotation appear to be extremely aggressive in their frequency. SQS charges per request and the Spring Cloud AWS defaults seem to be around 16 requests per second. I would like to use long pollling and adjust the interval. If anyone can point me to where I can set that I'd appreciate it. Obviously this can easily be done by using the Amazon SDK directly but I'm hoping there's a way to do this within Spring.
    Danilo Gomes

    @kanderson450 Thanks for your attention. Actually I have the same need but in order to increase or decrease rps according to queue (it'll be decided by profile, each one with its configurations). I would like to hardly increase the number of consumers to deal with 150 rps.

    If I'm right, I should pass other instance of AsyncTaskExecutor to SimpleMessageListenerContainer

    But, is this the right way?

    Spring Cloud AWS Buildmaster
    You should use the wait-time-out property on the listener container <aws-messaging:annotation-driven-queue-listener wait-time-out="60" />
    This defines the time the poller has to wait when no messages are in the queue (long polling), this is actually a feature provided by the AmazonSQS client.
    If you use Java Config you can set that property using the SimpleMessageListenerContainerFactory
    Danilo Gomes

    Thanks, guys. But that isn't my real question. Let's suppose that my application is consuming 50 records per second. If I want scale up to 150 rps, the only option that I have is scaling horizontally or can I increase (and control) my number of consumers?

    protected AsyncTaskExecutor createDefaultTaskExecutor() {
            String beanName = getBeanName();
            ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
            threadPoolTaskExecutor.setThreadNamePrefix(beanName != null ? beanName + "-" : DEFAULT_THREAD_NAME_PREFIX);
            int spinningThreads = this.getRegisteredQueues().size();
            if (spinningThreads > 0) {
                threadPoolTaskExecutor.setCorePoolSize(spinningThreads * DEFAULT_WORKER_THREADS);
                int maxNumberOfMessagePerBatch = getMaxNumberOfMessages() != null ? getMaxNumberOfMessages() : DEFAULT_WORKER_THREADS;
                threadPoolTaskExecutor.setMaxPoolSize(spinningThreads * maxNumberOfMessagePerBatch);
            // No use of a thread pool executor queue to avoid retaining message to long in memory
            return threadPoolTaskExecutor;

    Can I do it just overwriting AsyncTaskExecutor configuration?

    That is a citation of SimpleMessageListenerContainer.java
    Danilo Gomes
    Can you see what I'm facing? Does it make sense?
    Ryan Gardner
    @danilocgomes_twitter I ran into the same problem about not being able to easily ramp up the number of concurrent executors for the message listener so I ended up switching out my message polling and using camel instead
    for the others asking… if you want to do something with adding long polling, here’s what I put in my app’s config:
        public QueueBufferConfig queueBufferConfig(@Value("${QUEUE_CONSUMER_COUNT}") int consumerCount) {
            QueueBufferConfig config = new QueueBufferConfig();
            // use long polling to avoid going nuts with empty fetches
            // more in-flight batches can help maximize throughput per box
            config.setMaxInflightReceiveBatches(consumerCount * 2);
            return config;
        @Bean(destroyMethod = "shutdown")
        public AmazonSQSAsync amazonSQS( QueueBufferConfig queueBufferConfig, RegionProvider regionProvider, AWSCredentialsProvider awsCredentialsProvider) {
            AmazonSQSAsyncClient amazonSQSAsyncClient;
            amazonSQSAsyncClient = new AmazonSQSAsyncClient(awsCredentialsProvider);
            if (regionProvider != null) {
            return new AmazonSQSBufferedAsyncClient(amazonSQSAsyncClient, queueBufferConfig);
    Is there a possibility to disable the ec2 metadata lookup during startup thats happening in the OnAwsCloudEnvironmentCondition?
    Kyle Anderson
    @sbuettner cloud.aws.stack.auto=false
    Danilo Gomes

    Hey @ryangardner! I'm so sorry for delay. I was in a really really critical project and I need to change my focus.
    And, I noted that this chat isn't very active.

    So, thank you so much for reply me. Seriously you need change your solution for Camel?
    Is there no native solution for spring cloud aws?

    I'll try overwrite AsyncTaskExecutor mentioned previously and I'll get back with the results.

    Danilo Gomes

    Hey @ryangardner , I should confess that I've also used camel :D

    Thanks for your suggestion!

    Andy Czerwinski
    How do people build and test locally their microservices if you use the spring-cloud-aws annotations in their code?
    Kyle Anderson
    @czerwina I don't have an elegant answer, other than using Profile specific credentials. For example, for local development, I have a set of credentials on my local machine that are tied to a completely separate account from production. Same thing for staging. AWS Account segregation is what I do.
    @czerwina This can introduce config drift over time, but that's the best I've got.
    Andy Czerwinski
    @kanderson450 so you need to deploy to aws for each developer environment? Don't think our infrastructure dudes would appreciate paying for the hardware even if cheap.
    We use redis, oracle xd, rabbitmq locally which translate to elasticated, rds and sqs
    Kyle Anderson
    @czerwina I'd rather take the hit then use completely different components. RabbitMQ is nothing like SQS (even with an abstraction layer), and there's a lot of configuration that is specific to each. That's more config drift than separate AWS environments. And I'm not suggesting you need to deploy to AWS for local testing. You can make use of SQS, S3 and other services while running your app on your laptop. This isn't perfect either but SQS messages cost a miniscule amount.
    Andy Czerwinski
    @kanderson450 didn't think of just exposing the services to the devs. U think via api gateway or vpn to the vpc?
    Kyle Anderson
    @czerwina Depends on the service as they are not all available as private VPC endpoints. For example, SQS is not but S3 is. Your custom application services should be able to run locally all at once using docker-compose.
    The infrastructure services that are AWS specific (e.g. S3, SQS, SNS) can be called remotely from your laptop with Profile specific credentials tied to a completely separate environment (or even a seperate account).
    Steve Nera
    Heya, I was taking a look at RDS configuration support for Aurora and noticed that support for it was explicitly added back in October 2015. I was testing read replica support and it wasn't detecting the read replicas I had created.
    From what I understand, Amazon very recently (in January specifically) updated aws-java-sdk-rds to add a field for properly obtaining Aurora read replicas. The existing API that spring-cloud-aws is using results in an empty list for read replica identifiers if you're using an Aurora cluster.
    Alain Sahli
    Thanks for mentioning that! We are going to look into it.
    Felipe Adorno
    Hy everione, I create a register to create all my aws queus using annotations is interesting make a PR for this?
    I create the SqsConsumer to this and I use the parameters to create a SQS queue
    public @interface SqsConsumer {
        String[] value() default {};
        String delaySeconds() default "0";
        String maximumMessageSize() default "262144";
        String messageRetentionPeriod() default "345600";
        String policy() default "";
        String receiveMessageWaitTimeSeconds() default "0";
        String redrivePolicy() default "";
        String visibilityTimeout() default "30";
    Kyle Anderson

    Are you saying that you want to create the SQS queue if it doesn't exist, wherever the @SqsListener is defined? Something like:

    @SqsListener(value = "myQueue", createQueue=true)
    public void queueListener() {}

    Any clarification you can provide would be great. Thanks.

    Felipe Adorno
    @andersonkyle I'll do this, thanks for reply :)
    Al Grant
    anyone here connected Intellij via the plug to AWS?
    Al Grant
    anyone here using AWS???
    Al Grant
    I have my webapp running in AWS beanstalk. Each time a connection is made it prints the IP address to the console - but its printing - is that because beanstalk is using a proxy or?
    Kyle Anderson
    @bigalnz Yes, definitely an AWS user but I can't say that I've used the IntelliJ plugin...
    Is your question specific to this repository?
    I know this is AWS, but has anyone use Spring-Cloud Zuul and Eureka with Kubernetes?
    Kyle Anderson
    @JonathanAaron Can't say that I have. Are you running into an issue?
    Gaurav Rawat
    Is there any plan to support dimension publication through spring cloud aws actuator . Specially with relation to instance id's in case of multiple ec2 nodes publishing metrics of the same app .
    Nagilla Venkatesh
    Hi all,
    Need spring boot and Amazon redshift integration sample code
    can anyone please help me
    Narayanan Raghavan
    We have an issue with ZuulProxy throwing socketexception every 10 mins on PCF/AWS. Anyone seen this error?
    Prashant Srivastav
    @nagillavenkatesh best to use aws lamda with redshift and integrate with Spring
    it will be rest call to lambda uri and AWS Lambda Amazon Redshift Database Loader
    Felipe Adorno
    Hi all, anyone knows how I can reduce costs using SQSListener? pooling increase my billing a lot