Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Sep 16 14:15
    mp911de closed #445
  • Sep 16 14:15

    mp911de on main

    Replace `EmitterProcessor` with… (compare)

  • Sep 16 13:37
    mp911de milestoned #445
  • Sep 16 13:37
    mp911de labeled #445
  • Sep 16 13:37
    mp911de opened #445
  • Sep 16 13:37

    mp911de on main

    Replace EmitterProcessor with S… (compare)

  • Sep 16 13:36
    mp911de milestoned #219
  • Sep 16 13:36
    mp911de labeled #219
  • Sep 16 13:36
    mp911de opened #219
  • Sep 14 14:04
    mp911de commented #410
  • Sep 11 00:42
    kaladhar-mummadi commented #412
  • Sep 10 11:45
    mp911de milestoned #245
  • Sep 10 11:45
    mp911de labeled #245
  • Sep 10 11:45
    mp911de labeled #245
  • Sep 10 11:45
    mp911de opened #245
  • Sep 10 11:39
    mp911de commented #440
  • Sep 10 11:13
    OlegDokuka commented #440
  • Sep 10 11:12
    OlegDokuka commented #440
  • Sep 10 09:01
    mp911de commented #440
  • Sep 09 14:37
    ascertrobw commented #244
harish vashist
@harishvashistha1:matrix.org
[m]
@mp911de yes I mean the same, string SQL = select * with lot of joins, ins, likes . Created using Java string builder
Do you mean r2dbc does not support this parsed string query execution?
yakom
@yakom:matrix.org
[m]
hello, can i ask questions about Micronaut + R2DBC setup here?
the documentation under https://micronaut-projects.github.io/micronaut-data/latest/guide/#r2dbc is regrettably not really elaborate.
yakom
@yakom:matrix.org
[m]

i have:
-set up micronaut-data-processor
-configured an r2dbc datasource in application.yml
-extended ReactorCrudRepository with @R2dbcRepository(dialect = POSTGRES)

but what i get is a missing r2dbcOperations bean which, looking at its code, should be instantiated internally.

what have i missed?

yakom
@yakom:matrix.org
[m]
solved, wrong micronaut-data-r2dbc artifact.
Eyal Ringort
@eyalrin

Hi,
I'm trying to use JSON data type to store json data in a specific column. I followed this guide, so saving and reading JSON works but now I get (I have a boolean column as well in a table):

Caused by: org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.lang.Byte] to type [boolean]

What can I do??

1 reply
If I don't create a R2dbcCustomConversions bean then JSON doesn't work but I don't get this error...
ThiemoFischer
@ThiemoFischer

hey there, did anyone ran into problems using the following stack for the automated testing stage in your ci?

r2dbc + mssql server + junit 5 + testcontainers + github actions

Up front: All tests run locally without any problems or if we swap for a maria db or mysql db with their respective connectors.
When using mssql server: occasionally one or two tests fail in a non reproducible fashion. If rerun, the workflow will most likely run though. We suspect a concurrency problem. however junit is explicitly configured to not use parallel execution. when setting up or clearing test data we wait for the db with .block(), but in the failing case monos or fluxes under test complete without retrieving the testdata.

6 replies
EugeneZakhno
@EugeneZakhno
2021-08-05 12:54:21.732  INFO 7144 --- [           main] .TpsPersonalCabinetServiceAppApplication : Started TpsPersonalCabinetServiceAppApplication in 5.819 seconds (JVM running for 6.92)
2021-08-05 12:54:21.734  INFO 7144 --- [           main] o.s.b.a.ApplicationAvailabilityBean      : Application availability state LivenessState changed to CORRECT
2021-08-05 12:54:21.735  INFO 7144 --- [           main] o.s.b.a.ApplicationAvailabilityBean      : Application availability state ReadinessState changed to ACCEPTING_TRAFFIC
2021-08-05 12:55:14.311 ERROR 7144 --- [actor-tcp-nio-1] a.w.r.e.AbstractErrorWebExceptionHandler : [9c6a9d3a-1]  500 Server Error for HTTP GET "/findbyphonenumber/+111"

org.springframework.core.convert.ConversionFailedException: Failed to convert from type [io.r2dbc.postgresql.PostgresqlRow] to type [java.lang.Boolean] for value 'PostgresqlRow{context=ConnectionContext{client=io.r2dbc.postgresql.client.ReactorNettyClient@7a6e7d77, codecs=io.r2dbc.postgresql.codec.DefaultCodecs@5849c63d, connection=PostgresqlConnection{client=io.r2dbc.postgresql.client.ReactorNettyClient@7a6e7d77, codecs=io.r2dbc.postgresql.codec.DefaultCodecs@5849c63d}, configuration=PostgresqlConnectionConfiguration{applicationName='r2dbc-postgresql', autodetectExtensions='true', compatibilityMode=false, connectTimeout=null, database='intervale', extensions=[], fetchSize=io.r2dbc.postgresql.PostgresqlConnectionConfiguration$Builder$$Lambda$622/0x0000000800e623e0@7fcde469, forceBinary='false', host='10.2.84.132', loopResources='null', options='{search_path=personalcabinet}', password='*********', port=5432, preferAttachedBuffers=false, socket=null, tcpKeepAlive=false, tcpNoDelay=true, username='intervale'}, portalNameSupplier=io.r2dbc.postgresql.DefaultPortalNameSupplier@2f20bd94   .... bla...bla ...bla
EugeneZakhno
@EugeneZakhno
I'm trying to use method Mono<Boolean> existsById(ID id);
public interface UsersRepository extends ReactiveCrudRepository<User, Long> {
    @Query("SELECT phone_number FROM personalcabinet.users WHERE phone_number= :phoneNumber")
    Mono<Boolean> existsPhoneNumber(String phoneNumber);
below my controller:
   @GetMapping("/findbyphonenumber/{phoneNumber}")
    Mono<Boolean> existsPhoneNumber(@PathVariable String phoneNumber){
        return usersRepository.existsPhoneNumber(phoneNumber);
    }
1 reply
Help me, please. Say reason
EugeneZakhno
@EugeneZakhno
Maybe it works just for id because it's type Long? It's bad. We need more methods for finding from database
Dan Cohen-Smith
@dancohensmith
any known issues with the connection pool ? service is fine in normal running but if we put it under heavy load ( approx 20-25tps) we start to gt failed to acquire connection is 2000ms. we have turned on eviction and max-idle using remote depth to validate the connection. any ideas?
Umesh Karan
@umeshonlyumesh:matrix.org
[m]
Hi Guys ..I am facing problem while connecting oracle using r2dbc
this publisher does not support multiple subscribers
any idea ? how to fix this issue ? I am connecting to the remote oracle server
Mark Paluch
@mp911de
Umesh Karan
@umeshonlyumesh:matrix.org
[m]
thanks :)
Umesh Karan
@umeshonlyumesh:matrix.org
[m]
after taking the latest r2dbc driver ....that multiple subscriber issue gone but facing another issue
Failed to obtain R2DBC Connection; nested exception is io.r2dbc.spi.R2dbcTransientResourceException: [12514] [08006] Listener refused the connection with the following error: ORA-12514, TNS:listener does not currently know of service requested in connect descriptor (CONNECTION_ID=T8/zWBdvTgWge+ezncbOUg==)
org.springframework.dao.DataAccessResourceFailureException: Failed to obtain R2DBC Connection; nested exception is io.r2dbc.spi.R2dbcTransientResourceException: [12514] [08006] Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor
(CONNECTIONID=T8/zWBdvTgWge+ezncbOUg==)
at org.springframework.r2dbc.connection.ConnectionFactoryUtils.lambda$getConnection$0(ConnectionFactoryUtils.java:88)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
|
checkpoint ⇢ Handler com.example.demo.StudentController#getAllTaskStateEnum() [DispatcherHandler]
| checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
|
checkpoint ⇢ HTTP GET "/students/getAllTaskStateEnum" [ExceptionHandlingWebHandler]
Umesh Karan
@umeshonlyumesh:matrix.org
[m]
I think the problem sid v/s service name
can some help ? how to pass sid in connection url ?
Umesh Karan
@umeshonlyumesh:matrix.org
[m]
i have changed my connection url to use sid not service name and it worked
thanks
Alan Moffet
@amoffet
I have a question about the maven group id for the r2dbc postgresql driver. The readme in github shows that the group id is org.postgresql and the artifact id is r2dbc-postgresql. The latest release for this is 0.9.0.M1 - dated February 2021. However, io.r2dbc is also a group id in maven - the latest release being 0.8.8.RELEASE (May 2021). I'm curious about the track. It appears that I should anticipate 0.9.0.M2 for org.postgresql and that io.r2dbc is deprecated - but was released after the movement to org.postgresql for compatibility's sake? Or is there something else I should be aware of?
Umesh Karan
@umeshonlyumesh:matrix.org
[m]
can some help me here
no error but not showing the output
postman request is spinning
Robert Heim
@RobertHeim
Hi any idea why Json.of(myString) is not able to convert umlauts properly when run in a docker container (build with ./gradlew bootBuildImage) ? From the IDE it works fine.
Robert Heim
@RobertHeim
Turns out setting the env var JAVA_OPTS to -Dfile.encoding=UTF8 fixes it
kitkars
@kitkars
Hi Guys,
I would like to convert a field (column which is int type) to an Enum ?
We do not have the support for @Enumerated in r2dbc. is there any other option?
kitkars
@kitkars
Robert Heim
@RobertHeim
Does anyone know why the port is added twice when using SPRING_R2DBC_URL: r2dbc:postgresql://db_app:5432/app? It results in "cannot connect to db_app:5432:5432" help would be very appreciated! I posted more details here https://stackoverflow.com/questions/68571926/r2dbc-connection-port-added-twice-cannot-connect-to-db-app54325432-while-jdb
StalkerSun
@StalkerSun
Hello everyone! Can you please tell me how you can redefine the execution context for r2dbc to fill requests on boundElastic?
Shashank
@shankyty
Hello everyone,
I am looking for creating a sever which can be connected using jdbc and behave like mysql or postgres server.
Reason to do so is the client is not my code, and that platform will running production env. All I can do is can configure a mysql or postgres servers to pull data.
I have a code which generates data on fly. I want to stream that data to this platform.
aditya parikh
@adityaparikh_twitter
@RobertHeim I also saw that. Seems like a bug. It kept adding the 5432 default port even if I specified another port.
Anton Duyun
@Squiry
Can you reproduce it without spring?
Alexei Gorbachenko
@agorbachenko
Hi there! I have noticed that after r2dbc repository call my reactive chain was switching to be executed on netty event loop threads (reactor-tcp-<TRANSPORT>-N). This affects overall application's performance. Is this expected behavior? If so, I think it should be fixed in my code by switching to different scheduler (using .publishOn(...)) after every r2dbc query. Am I right?
Mark Paluch
@mp911de
Thread switching happens arbitrarily by default and threads shouldn't become a concern unless you run CPU-heavy workloads (e.g. JSON de-/serialization, crypto).
Typically, you have as many I/O threads as CPU cores. Depending on your usage scheme (one/many connections), you might observe different performance profiles.
Alexei Gorbachenko
@agorbachenko

I actually have reactive chain like:

  • incoming http query (webflux)
  • several db queries one by one (r2dbc)
  • outgoing http query (webclient)
  • several db queries one by one (r2dbc)
  • respond with result

I have several thousands rps. Application is fully reactive (no blocking). Most reactive steps are performed on reactor-tcp-epoll threads and I was observing a lot of db connections (200-300) in pending state at that time. Latency of the whole chain is about 200ms.

But if I change scheduler with .publishOn() after every r2dbc call, latency is 50ms and no pending db connections in pool.

Alexei Gorbachenko
@agorbachenko

I have some update on the problem. I tried to increase load in 1.5 times and noticed increased latency from 50 ms to 300 ms (while CPU utilization was far from 100%) for 5 app instances 4 CPU per instance. BUT changing deployment configuration to 10 instances 2 CPU per instance helps to reduce latency to 50 ms. So the total amount of system resources is the same but latency is affected.

Additionally I discovered that only 1 netty thread is busy - reactor-tcp-epoll-1. Thread-dump of other netty threads is:
java.lang.Thread.State: RUNNABLE
at app//io.netty.channel.epoll.Native.epollWait(Native Method)
....

This can explain why 10x2CPU performes better than 5x4CPU (total size of db connections is the same).

My question is it possible to tune r2dbc or netty to use more netty threads? Thanks in advance.

Mark Paluch
@mp911de
A I/O thread is bound to a connection. Connections cannot make use of multiple threads. The only way to run workload across multiple threads (without using publishOn) is to use multiple connections.
Mark Paluch
@mp911de
@agorbachenko sorry, it seems I accidentally deleted your message here while wanting to copy a permalink. Can you file your finding as ticket in https://github.com/reactor/reactor-netty/issues?
Alexei Gorbachenko
@agorbachenko

ok, I'll create ticket.

I also have restored my previous message:

I digged into the problem and found something strange in reactor-netty. While debugging how PostgresqlConnection's channel (->ReactorNettyClient->reactor.netty.Connection->io.netty.Channel) is binded to eventLoop from eventLoopGroup I found that first channel was binded to reactor-tcp-nio-1 (1) but next channels were binded to localLoop which is actually reactor-tcp-nio-1 (2).

reactor.netty.resources.ColocatedEventLoopGroup:
@Override
public EventLoop next() {
if (localLoop.isSet()) {
return localLoop.get(); // (2)
}
return eventLoopGroup.next(); // (1)
}

For test purposes I tried to override class reactor.netty.resources.DefaultLoopResources in my application's classpath with the same class but without wrapping into ColocatedEventLoopGroup.

DefaultLoopResources#cacheNativeClientLoops:
EventLoopGroup cacheNativeClientLoops() {
EventLoopGroup eventLoopGroup = cacheNativeClientLoops.get();
if (null == eventLoopGroup) {
EventLoopGroup newEventLoopGroup = cacheNativeServerLoops(); // LoopResources.colocate(cacheNativeServerLoops());
if (!cacheNativeClientLoops.compareAndSet(null, newEventLoopGroup)) {
// Do not shutdown newEventLoopGroup as this will shutdown the server loops
}
eventLoopGroup = cacheNativeClientLoops();
}
return eventLoopGroup;
}

With this fix all netty reactor-tcp-epoll threads were utilized and performance significantly increased.