@cppwfs I am triggering the task from SCDf and expecting to pick the volumeMount which I have provided in server-deployment.yaml but Task is not picking the volume mount as default mentioned on server-deployment .
server-deployment.yaml has below configuration>
- name: scdf-server image: springcloud/spring-cloud-dataflow-server:2.6.0-RC1 imagePullPolicy: Always volumeMounts: - name: vol-arg mountPath: /mnt/scdf/ readOnly: true - name: database mountPath: /etc/secrets/database readOnly: true
contributions welcomelabel. Otherwise, if you have any particular area where you feel it needs improvement, you can open a new issue and work on them, too. The SCDF UI project is another area you can look for enhancements also.
Hi guys I have a custom source app, is a web app.
I want to use the Metrics, prometheus and grafana.
Im adding these to my custom Stream (web app)
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> </dependency> <dependency> <groupId>io.micrometer</groupId> <artifactId>micrometer-registry-prometheus</artifactId> </dependency> <dependency> <groupId>io.micrometer.prometheus</groupId> <artifactId>prometheus-rsocket-spring</artifactId> <version>1.0.0</version> </dependency>
When deploy my Stream Im using the following properties:
app.movie-web.management.metrics.export.prometheus.enabled=true app.movie-web.management.metrics.export.prometheus.rsocket.enabled=true app.movie-web.management.metrics.export.prometheus.rsocket.host=prometheus-rsocket-proxy app.movie-web.management.metrics.export.prometheus.rsocket.port=7001
movie-web app the
/actuator doesn't show the
it's not being displayed in the Grafana dashboard.
Am I missing something??
The auto-configuration should pick everything up and enable prometheus endpoint right?
movie-web | splitter | groovy-transform | jdbcbut only the app-starters are showing up in the grafana dashboard, but no the
app-starters-micrometer-commondependency on your custom application. You can use one of the monitoring samples to follow-along and apply the similar dependency chain to get it up and running with Prom + Grafana.
Hey @sabbyanandan For the gateway timeout issue, I changed the ingress Nginx timeout to 3mins. Still getting the following issue intermittently in scheduler service -
Also, the pods which get this issue return an error in about 20 seconds.
Caused by: org.springframework.cloud.dataflow.rest.client.DataFlowClientException: GATEWAY_TIMEOUT at org.springframework.cloud.dataflow.rest.client.VndErrorResponseErrorHandler.handleError(VndErrorResponseErrorHandler.java:74) ~[spring-cloud-dataflow-rest-client-2.4.0.RELEASE.jar!/:2.4.0.RELEASE]
Also in our SCDF Server logs, we are getting this issue intermittently as well -
2020-08-06 09:01:54.965 WARN 1 --- [-nio-80-exec-10] o.h.engine.jdbc.spi.SqlExceptionHelper : SQL Error: 0, SQLState: null 2020-08-06 09:01:54.966 ERROR 1 --- [-nio-80-exec-10] o.h.engine.jdbc.spi.SqlExceptionHelper : HikariPool-1 - Connection is not available, request timed out after 30000ms. 2020-08-06 09:01:54.969 ERROR 1 --- [-nio-80-exec-10] o.s.c.d.s.c.RestControllerAdvice : Caught exception while handling a request org.springframework.dao.DataAccessResourceFailureException: unable to obtain isolated JDBC connection; nested exception is org.hibernate.exception.JDBCConnectionException: unable to obtain isolated JDBC connection at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:277)
Somehow it's unable to connect to DB. Do you think these could be related? Or any advice here?
@siddhantsorann: Ah, possibly related. You can change the datasource settings to have a bigger timeout value, but you may want to check why the DB connection is unstable.
It'd really also help to understand what is spitting out
GATEWAY_TIMEOUT and why; perhaps there's more stacktrace to it with the DEBUG enabled for Spring's REST template.
Hi @sabbyanandan & @cppwfs - Two issues with composed task runner 2.6.0.RC1 -
1- When you run a composed task , It is sometimes creating the black app execution .
2- It is getting hung at any app level and not moving on. Neither it is saying anything or any error in SCDF logs. It is also not starting the app so we can see the logs at app level. Atttached screen print for two composed task to show both the cases.
@cppwfs - Can you please advise me here? is it a know issue or we can log a dehect for this?
@ab48917: First, please switch to use 2.6 GA release; it went live last week. Secondly, there are a lot of variables that we don't have visibility into. For example, where you running, which platform, how you running, and what apps are in the CTR, etc.
It is hard to grok it in an ad-hoc chat format, without much context. I'd suggest writing the step-by-step details either in a gist or as a README in your GH repo. We can review your setup and attempt to reason through what could be causing the hold up in your box.