Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Spring Cloud AWS Buildmaster
    This defines the time the poller has to wait when no messages are in the queue (long polling), this is actually a feature provided by the AmazonSQS client.
    If you use Java Config you can set that property using the SimpleMessageListenerContainerFactory
    Danilo Gomes

    Thanks, guys. But that isn't my real question. Let's suppose that my application is consuming 50 records per second. If I want scale up to 150 rps, the only option that I have is scaling horizontally or can I increase (and control) my number of consumers?

    protected AsyncTaskExecutor createDefaultTaskExecutor() {
            String beanName = getBeanName();
            ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
            threadPoolTaskExecutor.setThreadNamePrefix(beanName != null ? beanName + "-" : DEFAULT_THREAD_NAME_PREFIX);
            int spinningThreads = this.getRegisteredQueues().size();
            if (spinningThreads > 0) {
                threadPoolTaskExecutor.setCorePoolSize(spinningThreads * DEFAULT_WORKER_THREADS);
                int maxNumberOfMessagePerBatch = getMaxNumberOfMessages() != null ? getMaxNumberOfMessages() : DEFAULT_WORKER_THREADS;
                threadPoolTaskExecutor.setMaxPoolSize(spinningThreads * maxNumberOfMessagePerBatch);
            // No use of a thread pool executor queue to avoid retaining message to long in memory
            return threadPoolTaskExecutor;

    Can I do it just overwriting AsyncTaskExecutor configuration?

    That is a citation of SimpleMessageListenerContainer.java
    Danilo Gomes
    Can you see what I'm facing? Does it make sense?
    Ryan Gardner
    @danilocgomes_twitter I ran into the same problem about not being able to easily ramp up the number of concurrent executors for the message listener so I ended up switching out my message polling and using camel instead
    for the others asking… if you want to do something with adding long polling, here’s what I put in my app’s config:
        public QueueBufferConfig queueBufferConfig(@Value("${QUEUE_CONSUMER_COUNT}") int consumerCount) {
            QueueBufferConfig config = new QueueBufferConfig();
            // use long polling to avoid going nuts with empty fetches
            // more in-flight batches can help maximize throughput per box
            config.setMaxInflightReceiveBatches(consumerCount * 2);
            return config;
        @Bean(destroyMethod = "shutdown")
        public AmazonSQSAsync amazonSQS( QueueBufferConfig queueBufferConfig, RegionProvider regionProvider, AWSCredentialsProvider awsCredentialsProvider) {
            AmazonSQSAsyncClient amazonSQSAsyncClient;
            amazonSQSAsyncClient = new AmazonSQSAsyncClient(awsCredentialsProvider);
            if (regionProvider != null) {
            return new AmazonSQSBufferedAsyncClient(amazonSQSAsyncClient, queueBufferConfig);
    Is there a possibility to disable the ec2 metadata lookup during startup thats happening in the OnAwsCloudEnvironmentCondition?
    Kyle Anderson
    @sbuettner cloud.aws.stack.auto=false
    Danilo Gomes

    Hey @ryangardner! I'm so sorry for delay. I was in a really really critical project and I need to change my focus.
    And, I noted that this chat isn't very active.

    So, thank you so much for reply me. Seriously you need change your solution for Camel?
    Is there no native solution for spring cloud aws?

    I'll try overwrite AsyncTaskExecutor mentioned previously and I'll get back with the results.

    Danilo Gomes

    Hey @ryangardner , I should confess that I've also used camel :D

    Thanks for your suggestion!

    Andy Czerwinski
    How do people build and test locally their microservices if you use the spring-cloud-aws annotations in their code?
    Kyle Anderson
    @czerwina I don't have an elegant answer, other than using Profile specific credentials. For example, for local development, I have a set of credentials on my local machine that are tied to a completely separate account from production. Same thing for staging. AWS Account segregation is what I do.
    @czerwina This can introduce config drift over time, but that's the best I've got.
    Andy Czerwinski
    @kanderson450 so you need to deploy to aws for each developer environment? Don't think our infrastructure dudes would appreciate paying for the hardware even if cheap.
    We use redis, oracle xd, rabbitmq locally which translate to elasticated, rds and sqs
    Kyle Anderson
    @czerwina I'd rather take the hit then use completely different components. RabbitMQ is nothing like SQS (even with an abstraction layer), and there's a lot of configuration that is specific to each. That's more config drift than separate AWS environments. And I'm not suggesting you need to deploy to AWS for local testing. You can make use of SQS, S3 and other services while running your app on your laptop. This isn't perfect either but SQS messages cost a miniscule amount.
    Andy Czerwinski
    @kanderson450 didn't think of just exposing the services to the devs. U think via api gateway or vpn to the vpc?
    Kyle Anderson
    @czerwina Depends on the service as they are not all available as private VPC endpoints. For example, SQS is not but S3 is. Your custom application services should be able to run locally all at once using docker-compose.
    The infrastructure services that are AWS specific (e.g. S3, SQS, SNS) can be called remotely from your laptop with Profile specific credentials tied to a completely separate environment (or even a seperate account).
    Steve Nera
    Heya, I was taking a look at RDS configuration support for Aurora and noticed that support for it was explicitly added back in October 2015. I was testing read replica support and it wasn't detecting the read replicas I had created.
    From what I understand, Amazon very recently (in January specifically) updated aws-java-sdk-rds to add a field for properly obtaining Aurora read replicas. The existing API that spring-cloud-aws is using results in an empty list for read replica identifiers if you're using an Aurora cluster.
    Alain Sahli
    Thanks for mentioning that! We are going to look into it.
    Felipe Adorno
    Hy everione, I create a register to create all my aws queus using annotations is interesting make a PR for this?
    I create the SqsConsumer to this and I use the parameters to create a SQS queue
    public @interface SqsConsumer {
        String[] value() default {};
        String delaySeconds() default "0";
        String maximumMessageSize() default "262144";
        String messageRetentionPeriod() default "345600";
        String policy() default "";
        String receiveMessageWaitTimeSeconds() default "0";
        String redrivePolicy() default "";
        String visibilityTimeout() default "30";
    Kyle Anderson

    Are you saying that you want to create the SQS queue if it doesn't exist, wherever the @SqsListener is defined? Something like:

    @SqsListener(value = "myQueue", createQueue=true)
    public void queueListener() {}

    Any clarification you can provide would be great. Thanks.

    Felipe Adorno
    @andersonkyle I'll do this, thanks for reply :)
    Al Grant
    anyone here connected Intellij via the plug to AWS?
    Al Grant
    anyone here using AWS???
    Al Grant
    I have my webapp running in AWS beanstalk. Each time a connection is made it prints the IP address to the console - but its printing - is that because beanstalk is using a proxy or?
    Kyle Anderson
    @bigalnz Yes, definitely an AWS user but I can't say that I've used the IntelliJ plugin...
    Is your question specific to this repository?
    I know this is AWS, but has anyone use Spring-Cloud Zuul and Eureka with Kubernetes?
    Kyle Anderson
    @JonathanAaron Can't say that I have. Are you running into an issue?
    Gaurav Rawat
    Is there any plan to support dimension publication through spring cloud aws actuator . Specially with relation to instance id's in case of multiple ec2 nodes publishing metrics of the same app .
    Nagilla Venkatesh
    Hi all,
    Need spring boot and Amazon redshift integration sample code
    can anyone please help me
    Narayanan Raghavan
    We have an issue with ZuulProxy throwing socketexception every 10 mins on PCF/AWS. Anyone seen this error?
    Prashant Srivastav
    @nagillavenkatesh best to use aws lamda with redshift and integrate with Spring
    it will be rest call to lambda uri and AWS Lambda Amazon Redshift Database Loader
    Felipe Adorno
    Hi all, anyone knows how I can reduce costs using SQSListener? pooling increase my billing a lot
    hi, how can I connect to an Aurora DB cluster replica using the spring-cloud-aws-jdbc? This is my java config: @EnableRdsInstance(dbInstanceIdentifier = "", databaseName = "", username = "", password = "", readReplicaSupport = true).
    On my JPA repository I'm using the @Transactional(readOnly=true).
    The documentation says that the library automatically detects the replica information, but it is not working for me. Additionaly, I think that the dbInstanceIdentifier should be the cluster name instead of the master instance.
    Sanjay Rawat
    Hi everyone, I am getting below error when I start the application. I have one SQS Listener with default config, I am also getting the message from Queue, but this error keeps popping up:
        Exception in thread "XMLGeneratorTaskExecutor-2" java.lang.StringIndexOutOfBoundsException: String index out of range: -1
                at java.lang.String.substring(String.java:1931)
                at org.springframework.cloud.aws.messaging.core.QueueMessageUtils.getNumberValue(QueueMessageUtils.java:93)
                at org.springframework.cloud.aws.messaging.core.QueueMessageUtils.getMessageAttributesAsMessageHeaders(QueueMessageUtils.java:80)
                at org.springframework.cloud.aws.messaging.core.QueueMessageUtils.createMessage(QueueMessageUtils.java:56)
                at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$MessageExecutor.getMessageForExecution(SimpleMessageListenerContainer.java:375)
                at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$MessageExecutor.run(SimpleMessageListenerContainer.java:336)
                at org.springframework.cloud.aws.messaging.listener.SimpleMessageListenerContainer$SignalExecutingRunnable.run(SimpleMessageListenerContainer.java:392)
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                at java.lang.Thread.run(Thread.java:748)
    Sanjay Rawat
    Also, is anyone able to get the RDS with PostgreSql working with Spring Cloud AWS?
    I am getting error with that as well, Tomcat JDBC pool keeps looking for Mysql driver, even after I set the driverClassName to org.postgresql.Driver
    Maciej Walkowiak
    Hey everyone! I've been playing recently with Spring Cloud AWS ElastiCache support for Redis. I am wondering why it doesn't use RedisCacheManager but SimpleCacheManager instead? I was expecting that it will works just as normal Redis backed cache, but the behaviour is different.