Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Glenn Renfro
    @cppwfs
    Thanks for asking!
    robdickens
    @robdickens
    Hi, I have deployed spring cloud data flow (2.2.0.RELEASE) onto an open source cloud foundry and I am having trouble with the composed task. I can't seem to override the "localhost:9393" default in composed task runner. Can anyone highlight the environment variable I should be using - preferably the one on Data Flow so that all tasks use the same Data Flow server URI. I have tried 'SPRING_CLOUD_DATAFLOW_CLIENT_SERVER_URI' and 'DATAFLOW_SERVER_URI' on my data flow server envs but neither are working for me.
    Glenn Renfro
    @cppwfs
    robdickens
    @robdickens
    @cppwfs Thank you!!!! I have to admit, between https://dataflow.spring.io/docs/ & https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/ I've been a bit overwhelmed, can I assume both are still in play currently?
    Glenn Renfro
    @cppwfs
    Yes.
    Think about https://dataflow.spring.io/docs/ shows what you can do with Data Flow (Stream and Task) and how to do it.
    And https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/ are the detailed instructions.
    robdickens
    @robdickens
    Perfect. Thanks @cppwfs for your fast responses, much appreciated!
    Glenn Renfro
    @cppwfs
    Glad I could help
    Dmytro Mykhailov
    @RaccoonDev
    Hello, everybody. I've just started with Spring Cloud Dataflow. Following getting started guide I'm running server and skipper with rabbitmq on my laptop. Trying to run timestamp example from getting started guide failing for me. From timestamp task logs I see error: java.lang.IllegalArgumentException: Delegate listener must not be null. Any ideas/hints for fixing that?
    Glenn Renfro
    @cppwfs
    Can you provide a full stack trace?
    Dmytro Mykhailov
    @RaccoonDev
    Sure.
    2019-08-13 18:57:17.727 WARN 32463 --- [ main] s.c.a.AnnotationConfigApplicationContext : Exception thrown from ApplicationListener handling ContextClosedEvent

    java.lang.IllegalArgumentException: Delegate listener must not be null
    at org.springframework.util.Assert.notNull(Assert.java:134) ~[spring-core-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.event.GenericApplicationListenerAdapter.<init>(GenericApplicationListenerAdapter.java:48) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.event.AbstractApplicationEventMulticaster.supportsEvent(AbstractApplicationEventMulticaster.java:287) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.event.AbstractApplicationEventMulticaster.retrieveApplicationListeners(AbstractApplicationEventMulticaster.java:236) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.event.AbstractApplicationEventMulticaster.getApplicationListeners(AbstractApplicationEventMulticaster.java:192) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:128) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:393) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:347) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:991) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:958) ~[spring-context-4.3.12.RELEASE.jar!/:4.3.12.RELEASE]
    at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:750) ~[spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:314) ~[spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118) ~[spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107) ~[spring-boot-1.5.8.RELEASE.jar!/:1.5.8.RELEASE]
    at org.springframework.cloud.task.app.timestamp.TimestampTaskApplication.main(TimestampTaskApplication.java:29) ~[classes!/:1.3.0.RELEASE]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
    at java.base/java.lang.reflect.Method.invoke(Method.java:567) ~[na:na]
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[timestamp-task-1.3.0.RELEASE.jar:1.3.0.RELEASE]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[timestamp-task-1.3.0.RELEASE.jar:1.3.0.RELEASE]
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) ~[timestamp-task-1.3.0.RELEASE.jar:1.3.0.RELEASE]
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) ~[timestamp-task-1.3.0.RELEASE.jar:1.3.0.RELEASE]
    Ha! Found that the latest version of the task is 2.1.0.RELEASE. Registered the version and ran the task again. Now works.
    Glenn Renfro
    @cppwfs
    sweet!
    Dmytro Mykhailov
    @RaccoonDev
    So, looks like some incompatibility with new version
    (going to check if there a way to suggest docs update)
    Glenn Renfro
    @cppwfs
    For your previous example did you build the 2.2.0 yourself?
    Dmytro Mykhailov
    @RaccoonDev
    nope
    downloaded jar files according to the instructions
    Glenn Renfro
    @cppwfs
    Can you share the link to the instructions?
    Dmytro Mykhailov
    @RaccoonDev
    The version of the task is shown here: https://dataflow.spring.io/docs/concepts/architecture/
    BTW, just created pull request for the version update
    Specified in the section related to batch processing
    Glenn Renfro
    @cppwfs
    @RaccoonDev found the issue. The 1.3 line of task apps does not work with Java 9+. So open a issue for the docs. And submit if a PR if you like :-)
    The issue should have that example use 2.1.0 instead of 1.3
    Dmytro Mykhailov
    @RaccoonDev
    Cool :) PR is there. Will open an issue when get back to laptop :)
    robdickens
    @robdickens
    Hi, we're using a composed task runner launched through data flow and when setting the properties of the child tasks we have come across the issue described here: https://stackoverflow.com/questions/48553059/spring-cloud-dataflow-task-properties-are-not-refreshed-between-executions where (if I understand correctly) if you use data flow and tasks on cloud foundry (which we are using the open source version of) then the properties are not being refreshed per run. Can anyone explain the correct method we should use to set runtime arguments on child tasks when launching a composed task runner through data flow?
    GamiiisKth
    @GamiiisKth
    Hi, I have deployed SCDF 2.2.0 to k8s. When I'm trying to deploy a stream i got the following exception java.lang.IllegalArgumentException: No platforms configured.
    Glenn Renfro
    @cppwfs
    @robdickens Sorry for the delay in the response. Use the composed-task-arguments argument. This option will allow you to specify the command line arguments that will be passed to all the child applications of a CTR execution. i.e. --composed-task-arguments=“foo=bar --baz=boo” will set the command line args for all child apps to foo=bar —baz=boo.
    Its documented in CTR but not in the current SCDF reference docs. We just added it to the latest snapshot, https://docs.spring.io/spring-cloud-dataflow/docs/current-SNAPSHOT/reference/htmlsingle/#_configuration_options.
    robdickens
    @robdickens
    Thanks @cppwfs , we will check this out tomorrow. That's really helpful!
    Sabby Anandan
    @sabbyanandan
    @GamiiisKth: Can you describe how you provisioned SCDF on K8s? Did you use the helm-chart or the deployment YAMLs? If the YAMLs directly, did you customized any of the defaults in SCDF or Skipper?
    venkatasreekanth
    @venkatasreekanth
    is there example for bulk importing apps using properties file in SCDF
    Sabby Anandan
    @sabbyanandan

    @venkatasreekanth: As far as the app coordinates are lined-up in the format as described here: https://docs.spring.io/spring-cloud-dataflow/docs/2.2.0.RELEASE/reference/htmlsingle/#_bulk_import_of_applications, you can point to the property file both from the Shell and the UI to import them into SCDF's App Registry.

    You can use any of the bulk import files that we ship for the out-of-the-box apps for reference also. Example: https://dataflow.spring.io/kafka-maven-latest

    robdickens
    @robdickens
    Hi @cppwfs , just an update, we tried you suggestion above (running CTR 2.1.0.RELEASE) and it didn't work as we expected.
    Using --composed-task-arguments=“foo=bar --baz=boo” did not pass through either and caused a strange split on the space within the composed task arguments.
    The form which worked for us was --composed-task-arguments=“--foo=bar,--baz=boo” (NB: no space either side of the comma) which perfectly passed through the arguments to the child arguments as well as allowing the configuration bindings to work where we were using a conditional property for our dependency injection.
    Not sure whether the docs are not quite right, or the code is not behaving as expected. Would be good to get your views on this.
    Thanks again for the continued help here.
    Glenn Renfro
    @cppwfs
    How did you launch your Composed Task? Shell, RestFul or UI?
    robdickens
    @robdickens
    @cppwfs used the shell to debug, tested with the UI and then finally RestFul which is how we have integrated
    @cppwfs FYI - I escaped the quotes for the composed task arguments when using the shell - guess that's obvious though
    Glenn Renfro
    @cppwfs
    Thanks for the heads up. I’ll look into this
    robdickens
    @robdickens
    No problem!
    JohannesM24
    @JohannesM24
    Hey @sobychacko I finally had the time and need to test the gradle metadata plugin https://github.com/spring-cloud/spring-cloud-app-starters-maven-plugins/tree/master/spring-cloud-app-starter-metadata-gradle-plugin . I have built it locally and successfully used it to create the companion artifact as we are now deploying via k8 and needed it. An official release plan of the plugin would be nice.
    Soby Chacko
    @sobychacko
    @JohannesM24 We will look into that.
    Need to create a CI plan first.
    GamiiisKth
    @GamiiisKth
    @sabbyanandan Hi I deployed scdf through the kubectl and, I did not customize default SCDF or Skipper
    java.lang.IllegalArgumentException: No platforms configured at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.determinePlatformName(SkipperStreamDeployer.java:285) at org.springframework.cloud.dataflow.server.stream.SkipperStreamDeployer.deployStream(SkipperStreamDeployer.java:220)
    Sabby Anandan
    @sabbyanandan
    Hi. One likely issue could be access to SCDF/Skipper's ConfigMap having issues in your setup. We recently we through a troubleshooting process at: spring-cloud/spring-cloud-dataflow#3439 - perhaps this might give you some ideas as to how to troubleshoot.
    Feel free to share SCDF and Skipper's kubectl describe <pod-name> output, so we can verify whether the default platform is configured correctly.
    PraneethPathireddy
    @PraneethPathireddy
    Hello, I have multiple datasources in my spring cloud task application. One datasource is used for application's business. I want to configure another datasource which will be used for Spring cloud tasks execution status. I read in the document that I have to configure datasource by providing custom TaskConfigurer. I have the following class for that
    @Configuration
    public class CustomTaskConfigurer extends DefaultTaskConfigurer {
    @Autowired
    public CustomTaskConfigurer(@Qualifier("secondDataSource") DataSource dataSource) {
    super(dataSource);
    }
    }
    But, I am getting the execption : java.lang.IllegalStateException: Expected one TaskConfigurer but found 2. Can anyone please help me out here
    Haruhiko Nishi
    @hanishi
    Hi, I am experiencing some weird issue when one of my stream app is deployed via SCDF server.
    Unexpected Content-Type=application/json is added to MessageHeader and this does not happen when the same app is just launched from my console.

    This happens when there is an app that uses the following annotation.

        @StreamListener(Processor.INPUT)
        @SendTo(Processor.OUTPUT)

    Content-Type=application/json only appears in the MessageHeader when deployed via SCDF. (Does not appear when it is launched manually in console)

    Haruhiko Nishi
    @hanishi
    Any clue?