These are chat archives for spring-cloud/spring-cloud

15th
Feb 2018
Abhishek Dujari
@abshkd
Feb 15 2018 02:07
how do i create and forward custom headers with Netflix Zuul based gateway? The client isnt sending them. I need to create them programmatically and forward to the services behind the gateway
can this be done with Security Filter?
Abhishek Dujari
@abshkd
Feb 15 2018 02:53
I was looking at ZuulFilter but not sure if it has a stage where I am forwarding after the security filters are applied
Spencer Gibb
@spencergibb
Feb 15 2018 03:14
@abshkd all zuul filters run after spring security
Abhishek Dujari
@abshkd
Feb 15 2018 03:56
@spencergibb Thank you! I will try to add auth headers to requests using zuulFilter and hopefully get it right
Should I be concerned with Ribbon @spencergibb or not necessary if I have the request headers in zuul?
David Welch
@dwelch2344
Feb 15 2018 04:26

Hey friends. Trying to get SpringCloud working in an existing Scala environment, primarily for service communication via RestTemplate. I don't want to boot an entire spring app (especially wrt to web) so I took a stab at a standalone AppCtx.

I've got it 99% of the way, with RestTemplate doing the substitution and everything – but in the end ribbon never can find clients (even when I've got consul pulled up and verified it should be there). It looks like RibbonClientConfiguration is superseding ConsulRibbonClientConfiguration - so rest template never actually pings consul.

I threw the relevant standalone bits in a gist – commit the project shortly too. Would love some thoughts if anyone else is neglecting their Valentine's date ;)

Just pushed it. The discovery branch has a BootTest that should be pretty easy to run and get an idea
Spencer Gibb
@spencergibb
Feb 15 2018 04:38
maybe don't add RibbonClientConfiguration in the list
SpringClientFactory from RibbonAutoConfiguration already references it.
David Welch
@dwelch2344
Feb 15 2018 04:42
no dice
tried process of elimination
David Welch
@dwelch2344
Feb 15 2018 05:31
Hmm yeah not sure. Another thing of interest: the AutoConfigs don't seem to pick up a lot of packages. only a few matches on the report
sur7725
@sur7725
Feb 15 2018 10:28

Discovery client name is not resolved in WebResource Object of RestClient. Whereas it is working fine with RestTemplate on registered discovery client name.

We are getting Exception com.sun.jersey.api.client.ClientHandlerException Occured during REST call.

com.sun.jersey.api.client.ClientHandlerException: java.net.UnknownHostException:. Please help

sur7725
@sur7725
Feb 15 2018 12:28

How can we read directory folder from any server. My requirement is that I know the server path where file can exist.

We need to load all the list of directory and list of files from the given path of the server i.e http://{serveripaddress}:{port}/{server-contextpath}/{folder-abc}.

Once I get the File handler (java.io.File) will be able to crawl or traverse to load list of files.

I am working on spring boot. Anyone can help on this?

dharezlak
@dharezlak
Feb 15 2018 15:07
Hello, I am trying to use spring-cloud-gateway 2.0.0.M6 and it looks like the Routes class mentioned here http://cloud.spring.io/spring-cloud-static/spring-cloud-gateway/2.0.0.M6/single/spring-cloud-gateway.html#_fluent_java_routes_api is missing. How can I configure the gateway routes without it?
Spencer Gibb
@spencergibb
Feb 15 2018 15:24
@dharezlak yeah, docs are outdated
dharezlak
@dharezlak
Feb 15 2018 15:26
@spencergibb Thanks. Another problem I am facing is testing with WireMock which has a dependency on Jetty.
...which leads to:
java.lang.ClassCastException: org.springframework.core.io.buffer.DefaultDataBufferFactory cannot be cast to org.springframework.core.io.buffer.NettyDataBufferFactory
    at org.springframework.cloud.gateway.filter.NettyWriteResponseFilter.lambda$filter$0(NettyWriteResponseFilter.java:61) ~[spring-cloud-gateway-core-2.0.0.M6.jar:2.0.0.M6]
    at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44) ~[reactor-core-3.1.3.RELEASE.jar:3.1.3.RELEASE]
    at reactor.core.publisher.Mono.subscribe(Mono.java:3006) ~[reactor-core-3.1.3.RELEASE.jar:3.1.3.RELEASE]
Is there a way to use WireMock with the gateway for testing?
Spencer Gibb
@spencergibb
Feb 15 2018 15:27
I don't know
dharezlak
@dharezlak
Feb 15 2018 15:29
Is there another way of stubbing a downstream server to test the gateway's behavior?
Spencer Gibb
@spencergibb
Feb 15 2018 15:46
we use @SpringBootTest internally
dharezlak
@dharezlak
Feb 15 2018 15:46
I used MockServer which uses netty under the hood but I am still getting the exception.
I am also using @SpringBootTest but my setup is the following: I have a Spring Boot app which is my gateway and during the tests I would like to start my Spring Boot gateway app and a third-party server to test my routing configurations.
I used WireMock for such scenarios so far but it requires Jetty which autoconfigures my gateway app to use it as well.
MockServer solves the Jetty dependency problem (no more Jetty in the classpath) but the ClassCastException persists.
Dave Syer
@dsyer
Feb 15 2018 15:53
You need to use the fat jar from wiremock with all its dependencies repackaged.
Maybe look at using spring-cloud-contract-wiremock. Then you get some extra features for free and no need to worry about the jetty thing.
dharezlak
@dharezlak
Feb 15 2018 16:03
@dsyer Thanks, but I am still getting the same error.
I can see in my logs that WireMock returned the response.
Spencer Gibb
@spencergibb
Feb 15 2018 16:04
you've got jetty or tomcat on your classpath still
dharezlak
@dharezlak
Feb 15 2018 16:04
No Jetty nor Tomcat
o.s.b.web.embedded.netty.NettyWebServer : Netty started on port(s): 33571
Tyler Van Gorder
@tkvangorder
Feb 15 2018 16:05

@khoubyari We turn off the configuration client and the discovery service for our unit tests via a TestPropertySource

@TestPropertySource(properties = {
        "eureka.client.enabled=false",
        "spring.cache.type=none",
        "spring.cloud.config.enabled=false"
    })
@ActiveProfiles(profiles = "unit")
@RunWith(SpringRunner.class)
public abstract class BaseTest implements ApplicationContextAware {

A lot of our devs use eclipse which does not separate test source from main source (that feature was just added to photon, finally).
Since our test sets the profile to "unit", we can insure it is only picked up in our tests by suffixing our configuration file with the profile: "/src/test/resources/application-unit.yml"

Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 16:22

@dsyer
Hi Team,
I have a use case where I need to create DataSource beans in BeanDefinitionRegistryPostProcessor. so to have them refreshed at runtime we have to create ScopedProxy for them explicitly.

Now if we use these data source bean in another BeanDefinitionRegistryPostProcessor , where I am registering DataSourceTransactionManager bean by setting this dataSource bean as constructor argument.

public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) {
        if (beanFactory instanceof BeanDefinitionRegistry) {
            BeanDefinitionRegistry registry = (BeanDefinitionRegistry) beanFactory;
            Map<String, DataSource> dataSources = beanFactory.getBeansOfType(DataSource.class);
            for (Map.Entry<String, DataSource> dsEntry : dataSources.entrySet()) {
                     DataSource ds = dsEntry.getValue();
                    if (ds instanceof ScopedObject) {
                registerBean(registry, dataSourceTransactionManagerName, ds,
                        DataSourceTransactionManager.class);
                            }
            }
        }
    }

    private void registerBean(BeanDefinitionRegistry beanDefRegistry, String dataSourceTransactionManagerName, DataSource dataSource, Class<?> beanTypeToBeInitialized) {

        BeanDefinitionBuilder bdb = BeanDefinitionBuilder.genericBeanDefinition(beanTypeToBeInitialized);
        bdb.addConstructorArgValue(dataSource);
            bdb.setScope("refresh");
        beanDefRegistry.registerBeanDefinition(dataSourceTransactionManagerName, bdb.getBeanDefinition());
    }

When I make any Db transaction like saving a record using JPA repository, it gives below error:
Caused by: java.lang.IllegalStateException: No value for key [org.apache.tomcat.jdbc.pool.DataSource@2126f417
at
org.springframework.transaction.support.TransactionSynchronizationManager.unbindResource(TransactionSynchronizationManager.java:210)

David Welch
@dwelch2344
Feb 15 2018 16:22
Hmmm anyone here well versed in the Ribbon initialization? I can't seem to marry up Consul + Ribbon, and I think it might have something to do with the clientId in ribbon not matching the ribbon.client.name not playing nice with consul
it seems like he's always looking for "client"
Dave Syer
@dsyer
Feb 15 2018 16:28
@JagmohanSharma can you distil that down to a simple sample app just using @RefreshScope (i.e. not your crazy post processor)?
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 16:29
Yes I did that. it works fine with @RefreshScope marked data source bean in a @Configuration class.
As when using @Configuration we have ConfigurationClassEnhancer and ConfigurationClassPostProcessor which replace them with CGLIB-enhanced subclasses.
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 16:42

as in TransactionSynchronizationManager's unbindResource() method when it execute

Object actualKey = TransactionSynchronizationUtils.unwrapResourceIfNecessary(key);

we get Cglib enhanced data source in case of @RefreshScope with configuration class, but not in case of post processor when it does
if (aopAvailable) { // now unwrap scoped proxy resourceRef = ScopedProxyUnwrapper.unwrapIfNecessary(resourceRef); } in unwrapResourceIfNecessary() method.

Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 17:37

basically we get different key at doUnbindResource() method in case of post processor approach where we are explicitly creating scopedProxy.

public static Object unbindResource(Object key) throws IllegalStateException {
        Object actualKey = TransactionSynchronizationUtils.unwrapResourceIfNecessary(key);
        `Object value = doUnbindResource(actualKey);`
        if (value == null) {
            throw new IllegalStateException(
                    "No value for key [" + actualKey + "] bound to thread [" + Thread.currentThread().getName() + "]");
        }
        return value;
    }

` private static Object doUnbindResource(Object actualKey) { Map<Object, Object> map = resources.get();

Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 17:44

@dsyer But this issue does not occur if when we set dataSource as shown below in DataSourceTransactionManager.

DataSource scopedDataSource = (DataSource) ((ScopedObject) ds).getTargetObject();

But we wont get DataSourceTransactionManager bean refreshed at runtime with this approach.

Dave Syer
@dsyer
Feb 15 2018 17:45
Why would you want the transaction manager to be refreshed?
It doesn't have any singleton state.
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 17:49
Like in case we set new value for DataSourceProperties like database URL is changed via the Environment.
Dave Syer
@dsyer
Feb 15 2018 17:50
But that isn't a property of the transaction manager is it? It's the data source.
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 17:53
yes agree.
Dave Syer
@dsyer
Feb 15 2018 17:55
It also looks like you only create one transaction manager in your little postprocessor, even though there might be multiple data sources?
So that looks odd to me
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 17:59
no, for each data source , we create transaction manager.
Map<String, DataSource> dataSources = beanFactory.getBeansOfType(DataSource.class);
            for (Map.Entry<String, DataSource> dsEntry : dataSources.entrySet()) {
                DataSource ds = dsEntry.getValue();
                if (ds instanceof ScopedObject) {
                    String dataSourceName = dsEntry.getKey();
                    DataSource scopedDataSource = (DataSource) ((ScopedObject) ds).getTargetObject();
                    registerBean(registry, dataSourceName, JDBC_TEMPLATE_NAME_SUFFIX, scopedDataSource,
                            JdbcTemplate.class);
                    registerBean(registry, dataSourceName, JDBC_TRANSACTION_MANAGER_NAME_SUFFIX, scopedDataSource,
                            DataSourceTransactionManager.class);
                }
            }
so whenever for example database URL is changed, we get new DataSource so we expect for transaction manager also use that.
Dave Syer
@dsyer
Feb 15 2018 18:03
I see. The snippet you pasted earlier just used the same bean name for all of them. I guess this is more complex.
Anyway, there's no need for a transaction manager to be in refresh scope, that I can see.
And I'm not sure I'm reading this right, but you definitely shouldn't be using the target data source in the constructor.
What would be the point of that?
You create a proxy and throw it away, basically, replacing it with its target.
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 18:08
DataSource scopedDataSource = (DataSource) ((ScopedObject) ds).getTargetObject();
actually if we do not pass target object here
And directly pass dsEntry.getValue() then we get that error
Caused by: java.lang.IllegalStateException: No value for key [org.apache.tomcat.jdbc.pool.DataSource@2126f417 at org.springframework.transaction.support.TransactionSynchronizationManager.unbindResource(TransactionSynchronizationManager.java:210)
Dave Syer
@dsyer
Feb 15 2018 18:10
None of this makes any sense to me.
I need some code I can run to figure it out
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 18:11
okk

And I'm not sure I'm reading this right, but you definitely shouldn't be using the target data source in the constructor.

Actually here did you mean that why we are doing

DataSource scopedDataSource = (DataSource) ((ScopedObject) ds).getTargetObject();

instead of directly using proxy object ds?

Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 18:37
when we direct use proxy object ds and pass that to DataSourceTransactionManager constructor then we get below error when we make any Db transaction like saving a record using JPA repository and resources are getting unbinded after commit in TransactionSynchronizationManager
Caused by: java.lang.IllegalStateException: No value for key [org.apache.tomcat.jdbc.pool.DataSource@2126f417 at org.springframework.transaction.support.TransactionSynchronizationManager.unbindResource(TransactionSynchronizationManager.java:210)
Jagmohan Sharma
@JagmohanSharma
Feb 15 2018 19:44

Anyway, there's no need for a transaction manager to be in refresh scope, that I can see.

But, should any change in DataSource not be used by transaction manager like when database URL is changed via Environment?