These are chat archives for spring-cloud/spring-cloud

28th
Feb 2018
Kent Johnson
@kentoj
Feb 28 2018 00:01
@spencergibb It looks like the CorsConfiguration is not being loaded by Spring WebFlux. It appears that adding a WebFluxConfigurer has no effect. What else would you suggest? Every This happens on any request from a JS client.
Haruhiko Nishi
@hanishi
Feb 28 2018 00:03
@dsyer
It was memory issue. So the shutdown was performed by container.
2018-02-28 08:59:56,033 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Process tree for container: container_e90_1519645367983_0029_01_000007 has processes older than 1 iteration running over the configured limit. Limit=536870912, current usage = 539611136
2018-02-28 08:59:56,033 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Container [pid=13170,containerID=container_e90_1519645367983_0029_01_000007] is running beyond physical memory limits. Current usage: 514.6 MB of 512 MB physical memory used; 2.4 GB of 1.0 GB virtual memory used. Killing container.
Dump of the process-tree for container_e90_1519645367983_0029_01_000007 :
        |- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
        |- 13170 13158 13170 13170 (bash) 0 0 115843072 356 /bin/bash -c /usr/java/latest/bin/java -Xms256m -Xmx512m  -Dserver.port=0 -Dendpoints.jmx.unique-names=true -Dspring.config.location=servers.yml -jar euler-httpclient-processor-kafka-10-1.0.0.BUILD-SNAPSHOT.jar --spring.cloud.application.group="euler-http-transfer" --spring.cloud.stream.metrics.properties="spring.application.name,spring.application.index,spring.cloud.application.*,spring.cloud.dataflow.*" --spring.cloud.dataflow.stream.name="euler-http-transfer" --spring.cloud.config.uri="http://10.203.0.21:8888" --spring.profiles.active="transfer" --spring.cloud.stream.bindings.output.destination="euler-http-transfer.euler-httpclient" --eureka.client.serviceUrl.defaultZone="http://10.203.0.21:8761/eureka,http://10.203.0.21:8762/eureka,http://10.203.0.21:8763/eureka" --spring.cloud.dataflow.stream.app.type="processor" --spring.cloud.dataflow.stream.app.label="euler-httpclient" --spring.cloud.stream.bindings.input.destination="all-iot-message" --spring.cloud.stream.bindings.output.producer.requiredGroups="euler-http-transfer" --spring.metrics.export.triggers.application.includes="integration**" --spring.cloud.stream.instanceCount="5" --spring.cloud.stream.metrics.key="euler-http-transfer.euler-httpclient.\${spring.cloud.application.guid}" --spring.cloud.stream.bindings.input.group="euler-http-transfer" 1>/yarn/container-logs/application_1519645367983_0029/container_e90_1519645367983_0029_01_000007/Container.stdout 2>/yarn/container-logs/application_1519645367983_0029/container_e90_1519645367983_0029_01_000007/Container.stderr
        |- 13199 13170 13170 13170 (java) 8379 156 2499334144 131385 /usr/java/latest/bin/java -Xms256m -Xmx512m -Dserver.port=0 -Dendpoints.jmx.unique-names=true -Dspring.config.location=servers.yml -jar euler-httpclient-processor-kafka-10-1.0.0.BUILD-SNAPSHOT.jar --spring.cloud.application.group=euler-http-transfer --spring.cloud.stream.metrics.properties=spring.application.name,spring.application.index,spring.cloud.application.*,spring.cloud.dataflow.* --spring.cloud.dataflow.stream.name=euler-http-transfer --spring.cloud.config.uri=http://10.203.0.21:8888 --spring.profiles.active=transfer --spring.cloud.stream.bindings.output.destination=euler-http-transfer.euler-httpclient --eureka.client.serviceUrl.defaultZone=http://10.203.0.21:8761/eureka,http://10.203.0.21:8762/eureka,http://10.203.0.21:8763/eureka --spring.cloud.dataflow.stream.app.type=processor --spring.cloud.dataflow.stream.app.label=euler-httpclient --spring.cloud.stream.bindings.input.destination=all-iot-message --spring.cloud.stream.bindings.output.producer.requiredGroups=euler-http-transfer --spring.metrics.export.triggers.application.includes=integration** --spring.cloud.stream.instanceCount=5 --spring.cloud.stream.metrics.key=euler-http-transfer.euler-httpclient.${spring.cloud.application.guid} --spring.cloud.stream.bindings.input.group=euler-http-transfer
Kent Johnson
@kentoj
Feb 28 2018 00:05
Ok, when I tried this filter then I got past the Invalid CORS problem. Now I have a new problem
Kent Johnson
@kentoj
Feb 28 2018 00:30
@Component
class MyCorsFilter : WebFilter {
    override fun filter(ctx: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
        if (ctx != null) {
            if (!ctx.response.headers.containsKey("Access-Control-Allow-Origin")) ctx.response.headers.add("Access-Control-Allow-Origin", "*")
            if (!ctx.response.headers.containsKey("Access-Control-Allow-Methods")) ctx.response.headers.add("Access-Control-Allow-Methods", "GET, PUT, POST, DELETE, OPTIONS")
            if (!ctx.response.headers.containsKey("Access-Control-Allow-Headers")) ctx.response.headers.add("Access-Control-Allow-Headers", "Authorization,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range")
            if (ctx.request.method == HttpMethod.OPTIONS) {
                if (!ctx.response.headers.containsKey("Access-Control-Max-Age")) ctx.response.headers.add("Access-Control-Max-Age", "1728000")
                ctx.response.statusCode = HttpStatus.NO_CONTENT
                return Mono.empty()
            } else {
                if (!ctx.response.headers.containsKey("Access-Control-Expose-Headers")) ctx.response.headers.add("Access-Control-Expose-Headers", "Authorization,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range")
                return chain?.filter(ctx) ?: Mono.empty()
            }
        } else {
            return chain?.filter(ctx) ?: Mono.empty()
        }
    }
}
It appears that the CORS headers were not being added to preflight OPTIONS requests by Spring WebFlux. This filter does the job (probably naively).
Abhishek Dujari
@abshkd
Feb 28 2018 02:46
do I need to add the okhttp dependency if I am switchign ribbon to it?
Spencer Gibb
@spencergibb
Feb 28 2018 02:47
yes
Abhishek Dujari
@abshkd
Feb 28 2018 02:48
thank you
please can you advise how i can force a timeout on forward requests?
my requests are going beyond 60 secodns
they tie up the threads
using zuul and hystrix edgware.sr2
ribbon:
    read-timeout: 1000
    okhttp:
        enabled: true
    eureka:
        enabled: true
# See http://cloud.spring.io/spring-cloud-netflix/spring-cloud-netflix.html
zuul: # those values must be configured depending on the application specific needs
    host:
        max-total-connections: 5000
        max-per-route-connections: 400
        socket-timeout-millis: 5000
        connect-timeout-millis: 300
        time-to-live: 5000
    ribbon-isolation-strategy: thread
    thread-pool:
# See https://github.com/Netflix/Hystrix/wiki/Configuration
hystrix:
    command:
        default:
            execution:
                isolation:
                    strategy: THREAD
                    thread:
                        timeoutInMilliseconds: 5000
Abhishek Dujari
@abshkd
Feb 28 2018 03:08
did I say something very wrong and stupid? i seem to get no answer for this. i found a few github issues and changed to okhttp. I've put in timeouts and what i could find but none of it has much effect
Kent Johnson
@kentoj
Feb 28 2018 03:10
You may need to enable timeouts:
hystrix:
  command:
    default:
      execution:
        timeout:
          enabled: true
        isolation:
          thread:
            timeoutInMilliseconds: ${timeouts.request-timeout-millis}
And the ribbon timeouts are different I think:
ribbon:
  ReadTimeout: 2000
  ConnectTimeout: 2000
Abhishek Dujari
@abshkd
Feb 28 2018 03:11
oh, ide seems to think its not CamelCased
is the hystrix.isolatio.streategy correct?
Kent Johnson
@kentoj
Feb 28 2018 03:13
not sure, I don't use that one.
Abhishek Dujari
@abshkd
Feb 28 2018 03:13
are you certain of the hystrix?
Kent Johnson
@kentoj
Feb 28 2018 03:13
Yes, it works for me
Abhishek Dujari
@abshkd
Feb 28 2018 03:13
great! thank you. ill try it now
ChrisConstantien
@ChriConstantine_twitter
Feb 28 2018 03:24
hi there
Abhishek Dujari
@abshkd
Feb 28 2018 03:48
the hystrix timeout seems to work, it does kick in and i can see zuul timeouts now
Kent Johnson
@kentoj
Feb 28 2018 03:48
Great, I'm glad I could help.
Abhishek Dujari
@abshkd
Feb 28 2018 03:48
indeed. you might have just saved my ass here
thank you very much
Abhishek Dujari
@abshkd
Feb 28 2018 04:34
this is new thing i have been meaning to ask
I assume stuff like threads for undertow etc can only be configured in the local configs. using config server will not magically increase threads
say if I add undertow.threads on config server it will not increase threads in the application even if I reload the ApplicationContext
or reboot the server
Kent Johnson
@kentoj
Feb 28 2018 07:26
@abshkd I don't know much about that though it sounds like it wouldn't increase threads in a running app, just config strings would be reloaded.
I think you can right your own custom config refresh triggers though to do that sort of thing.
at this point I think you would reboot the server.
salaboy
@salaboy
Feb 28 2018 08:32
hi everyone
quick question.. that someone might know from the top of their heads
I’m having issues with “org.springframework.cloud.context.restart.RestartEndpoint” this not being found
and I’m including spring-cloud-context in the project
Dave Syer
@dsyer
Feb 28 2018 08:33
"not found" in what sense?
salaboy
@salaboy
Feb 28 2018 08:34
it was being used into a @ConditionalBean
required a bean of type 'org.springframework.cloud.context.restart.RestartEndpoint' that could not be found.
do I need to instanciate it now in a configuration?
Dave Syer
@dsyer
Feb 28 2018 08:35
Nothing changed as far as I know
salaboy
@salaboy
Feb 28 2018 08:35
sorry it was used like this: @ConditionalOnClass({RestartEndpoint.class, ContextRefresher.class})
Dave Syer
@dsyer
Feb 28 2018 08:35
Do you have a sample app that shows what you did?
salaboy
@salaboy
Feb 28 2018 08:35
I’m working on that.. here: spring-cloud-incubator/spring-cloud-kubernetes#152
I’m trying to understand if I’m missing something or if something has changed..
as soon as I add sprint-cloud-context I was expecting that Class to be present and registered automatically, but my assumption might be wrong
Dave Syer
@dsyer
Feb 28 2018 08:36
Nothing changed. But conditionals are sensitive to ordering.
You might have just been lucky before.
A sample would help. Or a test in that project or something.
salaboy
@salaboy
Feb 28 2018 08:37
@dsyer I will work on that
thanks a lot
if nothing has changed it must be on my side
Haruhiko Nishi
@hanishi
Feb 28 2018 08:53
In the spring-cloud-netflix documentation where it discusses how ${random.value} can be used to make the whole string unique, in case when there is no ${spring.application.instance_id} is provided, but I find this should be avoided. When "refresh" is invoked on the instances that is refresh enabled, this value changes every time and gives you nothing but trouble. (ribbon client fails to hit its cache)
Dave Syer
@dsyer
Feb 28 2018 08:57
That's probably a good point. But discovery should be able to adapt.
Caches would be invalid of course
But that can happen anyway.
Can you be a bit more precise about what the issue is? Maybe find a good way to reproduce the problem?
Haruhiko Nishi
@hanishi
Feb 28 2018 08:58
Also it leaves a lot of DOWN(s)
in Eureka
Dave Syer
@dsyer
Feb 28 2018 08:59
A refresh can cause the service instance to need to re-register itself.
Haruhiko Nishi
@hanishi
Feb 28 2018 08:59
It's not a problem but it leads to mess in dashboard.
Dave Syer
@dsyer
Feb 28 2018 08:59
Maybe we can be smarter about when that is needed. But if you change the instance id, for sure it was needed.
The "mess" is just telling you what happened though.
So that's probably a good thing right?
Haruhiko Nishi
@hanishi
Feb 28 2018 09:01
That is also true.
Kent Johnson
@kentoj
Feb 28 2018 09:02
Ok, I finally got WebSockets proxying through Spring Cloud Gateway and an Nginx load balancer in front of a backend Spring Boot 2.0 service!
It is nice to finally have that done.
The hard part comes after this in figuring out how to maintain all the WebSocket connections.
and how to send messages to each user
Haruhiko Nishi
@hanishi
Feb 28 2018 09:05
problem is when the entries in the cache is mostly invalid after many refreshes
Dave Syer
@dsyer
Feb 28 2018 09:06
Seems like maybe the problem is too many refreshes.
So not everyone is going to experience that. (Hence it's the first we've heard of it I guess.)
Does your platform have an instanceid that is stable?
salaboy
@salaboy
Feb 28 2018 09:07
@kentoj interesting.. we are after the same problem.. can you share some links and pointers?
Dave Syer
@dsyer
Feb 28 2018 09:07
That's probably the best solution.
(As it already says in the ref guide)
Kent Johnson
@kentoj
Feb 28 2018 09:20
@Salaboy yes, after I get some sleep :) It's been a long day.
I'll be here over the next few days and weeks
Haruhiko Nishi
@hanishi
Feb 28 2018 09:23
I managed to find Env property for the yarn deployer, which is set to Env and it is container Id that the instance was launched. In our case, refresh is triggered when record of database changes and the change is made by our user. The record holds the destinations where http requests are forwarded by http client.
salaboy
@salaboy
Feb 28 2018 09:32
@kentoj :) please do share that whenever you feel more rested
@dsyer just for the record.. the problem with the restartEndpoint bean was the lack of spring-boot-actuator-autoconfigure moving the app from 1.5 to 2.x
just in case that someone else face the same problem
and I’m looking to find where this class went: Caused by: java.lang.ClassNotFoundException: org.springframework.security.rsa.crypto.RsaAlgorithm
Dave Syer
@dsyer
Feb 28 2018 09:53
Still there as far as I know
Bad jar?
salaboy
@salaboy
Feb 28 2018 09:56
it seems that in 2.0.0 the rsa has dissapeared
- org.springframework.cloud:spring-cloud-context:jar:2.0.0.BUILD-SNAPSHOT:compile
[INFO] | | - org.springframework.security:spring-security-crypto:jar:5.0.3.BUILD-SNAPSHOT:compile
let me dig deeper
Dave Syer
@dsyer
Feb 28 2018 09:58
5.0.3 looks like the wrong version
Where did that come from?
It's not managed by Boot any more.
But it is a dependency of spring-cloud-context. Which I expect you should be depending on?
salaboy
@salaboy
Feb 28 2018 10:00
yes I am.. but that has RSA as optional.. I think that this was the problem
that was it :)
sorry for all the bothering :)
Dave Syer
@dsyer
Feb 28 2018 10:03
That's probably a bug in the BOMs then
You shouldn't need to manage the version
Let me add it to spring-cloud-commons-dependencies
It's there already
salaboy
@salaboy
Feb 28 2018 10:03
it shoud be there.. I haven’t specified the version
Dave Syer
@dsyer
Feb 28 2018 10:04
You must have stuffed up your dependency management
salaboy
@salaboy
Feb 28 2018 10:04
at all but it was rsa was optional
Dave Syer
@dsyer
Feb 28 2018 10:04
No, you need to use a BOM.
Which project is this for?
that;’s what I needed in that pom to find the RsaAlgorithm class
I also think you need to clean that up a bit
Dave Syer
@dsyer
Feb 28 2018 10:07
spockframework cannot be needed by users?
salaboy
@salaboy
Feb 28 2018 10:07
yeah.. I’m that process of cleaning things up
Dave Syer
@dsyer
Feb 28 2018 10:08
Ditto rest-assured
salaboy
@salaboy
Feb 28 2018 10:08
so recommendations are more than welcome
Dave Syer
@dsyer
Feb 28 2018 10:08
spring-cloud-dependencies-parent is just for the plugin configuration
salaboy
@salaboy
Feb 28 2018 10:08
how do you handle that usually?
Dave Syer
@dsyer
Feb 28 2018 10:08
It doesn't have any dependency management
salaboy
@salaboy
Feb 28 2018 10:08
that’s true.. I was checking that
ok so I will do my homework on that front now
Dave Syer
@dsyer
Feb 28 2018 10:08
If you depend on spring-cloud-commons, normally you use the BOM from spring-cloud-commons, for instance
salaboy
@salaboy
Feb 28 2018 10:09
that at least is working
Dave Syer
@dsyer
Feb 28 2018 10:09
This stuff is hard to get right
Actually the right place to put this is in your parent pom
Dave Syer
@dsyer
Feb 28 2018 10:09
Not your BOM
Yeah that one
salaboy
@salaboy
Feb 28 2018 10:10
exactly
Dave Syer
@dsyer
Feb 28 2018 10:10
So that should work
If it's an optional dependency, it's harder for users
You want the RSA thing to be optional for your library, and for users, I guess?
salaboy
@salaboy
Feb 28 2018 10:11
it seems to be used inside the kubernetes projects.. so I would say that for spring-cloud-context is ok to be optional but not for spring-cloud-kubernetes
Dave Syer
@dsyer
Feb 28 2018 10:11
If it's not optional, it's much easier
You just add it to your pom and let Spring Cloud Commons manage the dependency
salaboy
@salaboy
Feb 28 2018 10:12
yeah.. done..
now about spockframework
should I move that to my parent pom instead of the bom right?
Dave Syer
@dsyer
Feb 28 2018 10:12
Probably
salaboy
@salaboy
Feb 28 2018 10:12
just to have it as dependencyMgmt and not push that to users
ok I will give that a try
Dave Syer
@dsyer
Feb 28 2018 10:13
Yes
salaboy
@salaboy
Feb 28 2018 10:13
I need to figure out why it was there in the first place
Dave Syer
@dsyer
Feb 28 2018 10:14
If you take it out and the tests are green I'd say that's a win
salaboy
@salaboy
Feb 28 2018 10:16
yeah.. tests are green
same for arquillian.. i think that the original authors wanted to push arquillian to be used for testing all over the place
but not sure if it is a wise decision
Abhishek Dujari
@abshkd
Feb 28 2018 15:28
my team is starting on graphql but it is ecpected that the gateway can do such things as resolving graphql
anyone has any idea about this and if spring cloud is working towards it?
salaboy
@salaboy
Feb 28 2018 15:40
@abshkd we are doing the same
in activiti-cloud
@abshkd you can join our channel.. if you want
Abhishek Dujari
@abshkd
Feb 28 2018 15:40
count me in
because google had nothing
Abhishek Dujari
@abshkd
Feb 28 2018 15:47
cant find your channel
lnxmad
@lnxmad
Feb 28 2018 16:41
hey guys, any resources here to help with a zuul question? I couldn't find another relevant channel
Dave Syer
@dsyer
Feb 28 2018 16:59
If you are using spring cloud then this is the right place
lnxmad
@lnxmad
Feb 28 2018 17:33
perfect! I guess what I'm trying to do with zuul is use PatternServiceRouteMapper to extract out a version from the url, but also define the routes manually. I dont necessarily want to build the service key from the routemapper, but get the version and when defining the routes manually, be able to define my service but also use the version
Kent Johnson
@kentoj
Feb 28 2018 19:57
@spencergibb I'd really love to create some documentation with examples for the Spring Cloud project. Now that I put all that effort into getting something working I'd like to make it easier for the next guy.
I have a working websockets app that goes from a standalone Angular 5 app through an NGINX proxy (for TLS termination), then through Spring Cloud Gateway, and finally to my destination server, all in Kotlin.
my email is my username @gmail.com so just ping me there and let's get this started!