Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 09:18
    mp911de commented #48
  • 09:18
    mp911de commented #48
  • 09:17
    mp911de commented #48
  • 09:17
    mp911de milestoned #48
  • 09:17

    mp911de on 0.8.x

    Update GCP Spanner to 0.5.0 … (compare)

  • 09:17
    mp911de closed #48
  • 09:16
    mp911de closed #46
  • 09:16

    mp911de on release-0.x

    Prepare next development iterat… Include oracle-r2dbc in BOM. [… Release Arabba-SR10 [closes #4… (compare)

  • 09:16

    mp911de on Arabba-SR10

    (compare)

  • 09:16

    mp911de on 0.8.x

    Release Arabba-SR10 [closes #4… Prepare next development iterat… (compare)

  • May 13 16:47
    ttddyy closed #93
  • May 13 16:43
    ttddyy closed #92
  • May 13 16:43
    ttddyy closed #91
  • May 13 08:22
    mp911de unlabeled #407
  • May 13 08:22
    mp911de labeled #407
  • May 13 06:38
    mp911de edited #46
  • May 13 02:28
    JoseLion edited #408
  • May 13 02:26
    JoseLion edited #408
  • May 13 02:26
    JoseLion edited #408
  • May 13 02:26
    JoseLion edited #408
capnrotbart
@capnrotbart
Hello there, I am having a sporadic back-pressure overflow specifically with r2dbc-mssql, anyone know if this is a bug or a user error?
I am specifically seeing this in integration tests where I do a blocking call, and I can reliably reproduce this under high CPU load. The use of EmitterProcessor in ReactorNettyClient seems to be issue, but I am not 100% sure.
reactor.core.Exceptions$OverflowException: Backpressure overflow during Sinks.Many#emitNext
    at reactor.core.Exceptions.failWithOverflow(Exceptions.java:233) ~[reactor-core-3.4.3.jar:3.4.3]
    Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Assembly trace from producer [reactor.core.publisher.MonoFlatMapMany] :
    reactor.core.publisher.Mono.flatMapMany
    io.r2dbc.mssql.client.ReactorNettyClient.exchange(ReactorNettyClient.java:591)
Error has been observed at the following site(s):
    |_       Mono.flatMapMany ⇢ at io.r2dbc.mssql.client.ReactorNettyClient.exchange(ReactorNettyClient.java:591)
    |_            Flux.handle ⇢ at io.r2dbc.mssql.client.ReactorNettyClient.exchange(ReactorNettyClient.java:591)
    |_  Flux.doAfterTerminate ⇢ at io.r2dbc.mssql.client.ReactorNettyClient.exchange(ReactorNettyClient.java:601)
    |_        Flux.doOnCancel ⇢ at io.r2dbc.mssql.client.ReactorNettyClient.exchange(ReactorNettyClient.java:601)
    |_     Flux.doOnSubscribe ⇢ at io.r2dbc.mssql.QueryMessageFlow.exchange(QueryMessageFlow.java:53)
    |_            Flux.handle ⇢ at io.r2dbc.mssql.QueryMessageFlow.exchange(QueryMessageFlow.java:54)
    |_                        ⇢ at io.r2dbc.mssql.util.Operators.discardOnCancel(Operators.java:43)
    |_         Flux.transform ⇢ at io.r2dbc.mssql.SimpleMssqlStatement.lambda$execute$0(SimpleMssqlStatement.java:135)
    |_       Flux.doOnDiscard ⇢ at io.r2dbc.mssql.SimpleMssqlStatement.lambda$execute$0(SimpleMssqlStatement.java:135)
    |_       Flux.windowUntil ⇢ at io.r2dbc.mssql.SimpleMssqlStatement.createResultStream(SimpleMssqlStatement.java:147)
    |_               Flux.mapat io.r2dbc.mssql.SimpleMssqlStatement.createResultStream(SimpleMssqlStatement.java:148)
    |_             Flux.defer ⇢ at io.r2dbc.mssql.SimpleMssqlStatement.execute(SimpleMssqlStatement.java:113)
    |_              Flux.fromat org.springframework.r2dbc.core.DefaultDatabaseClient$DefaultGenericExecuteSpec.lambda$execute$3(DefaultDatabaseClient.java:375)
    |_              Flux.castat org.springframework.r2dbc.core.DefaultDatabaseClient$DefaultGenericExecuteSpec.lambda$execute$3(DefaultDatabaseClient.java:376)
    |_             checkpoint ⇢ SQL "
                select * from something_with_more_than_250_rows
            " [DatabaseClient]
    |_           Flux.flatMap ⇢ at org.springframework.r2dbc.core.DefaultFetchSpec.lambda$all$2(DefaultFetchSpec.java:89)
    |_         Flux.usingWhen ⇢ at org.springframework.r2dbc.core.DefaultDatabaseClient.inConnectionMany(DefaultDatabaseClient.java:134)
    |_        Flux.onErrorMap ⇢ at org.springframework.r2dbc.core.DefaultDatabaseClient.inConnectionMany(DefaultDatabaseClient.java:146)
    |_                        ⇢ at org.springframework.r2dbc.core.DefaultFetchSpec.all(DefaultFetchSpec.java:87)
    |_ Flux.subscriberContext ⇢ at kotlinx.coroutines.reactor.ReactorContextInjector.injectCoroutineContex
    |_            Mono.createat kotlinx.coroutines.reactor.MonoKt.monoInternal(Mono.kt:50)
    |_                        ⇢ at kotlinx.coroutines.reactor.MonoKt.mono(Mono.kt:32)
    |_       Mono.subscribeOn ⇢ at company.TestSchedulerKt$runBlockingTest$1.invoke(TestScheduler.kt:18)
    |_         Mono.publishOn ⇢ at company.TestSchedulerKt$runBlockingTest$1.invoke(TestScheduler.kt:19)
        at reactor.core.Exceptions.failWithOverflow(Exceptions.java:233) ~[reactor-core-3.4.3.jar:3.4.3]
        at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:46) ~[reactor-core-3.4.3.jar:3.4.3]
        at reactor.core.publisher.EmitterProcessor.onNext(EmitterProcessor.java:265) ~[reactor-core-3.4.3.jar:3.4.3]
        at io.r2dbc.mssql.client.ReactorNettyClient$1.next(ReactorNettyClient.java:244) ~[r2dbc-mssql-0.8.5.RELEASE.jar:0.8.5.RELEASE]
        ...
Mark Paluch
@mp911de
@capnrotbart are you using a connection pool? Also, how does your code look like? According to the message, there seem to be more than 16 pending requests on the connection.
9 replies
Nilesh Sargar
@nsargar
Hi, I am new to r2dbs and I am trying to fetch data using projection (r2dbcTemplate.select.as()) but not able to apply the distinct, searched a lot but no luck, can anyone please point me in a direction. Thanks in advance.
Syed Mainul Hasan
@mainul35
I am migrating a project from spring-boot-starter-data-jpa to spring-boot-starter-data-r2dbc but cannot find @GeneratedValue, @PrePersist, @PreUpdate annotations
Mark Paluch
@mp911de
peter royal
@osi
Is there a reason why rollbackTransaction doesn't assert that the connection isn't closed in r2dbc-pool? https://github.com/r2dbc/r2dbc-pool/blob/1c54e515e151ca082b766713960197085ed3c8c5/src/main/java/io/r2dbc/pool/PooledConnection.java#L138
Mark Paluch
@mp911de
Probably an oversight. Care to submit a pull request for a fix?
Randula Prasad Dela
@randuladela
Can anyone help me with this error,I am using R2DBC for Oracle
java.lang.IllegalStateException: This publisher does not support multiple subscribers
I used Spring boot webflux
Mark Paluch
@mp911de
There's an open ticket: oracle/oracle-r2dbc#22
Randula Prasad Dela
@randuladela
thank you very much ,I am new to R2DBC sir
Michal Holič
@holdooo:matrix.org
[m]
Hello guys,
I'm wondering what's the status of supporting execution of stored procedures?
It's a feature which blocks adoption for us.
Mark Paluch
@mp911de
Postgresql, SQL Server and Oracle support running stored procedures. IIRC, H2 as well.
Michal Holič
@holdooo:matrix.org
[m]
Oh, it's listed in the "next steps" list on the r2dbc-mssql README.md, confusing.
Mark Paluch
@mp911de
The caveat is that while you can run stored procedures, you cannot consume OUT parameters yet, so from that perspective the support isn't complete.
peter royal
@osi
@mp911de aye, i'll add a PR for asserting closed on rollback methods. i wanted to check that the omission wasn't intentional first.
Mark Paluch
@mp911de
Thanks
mariaAbramiuc
@mariaAbramiuc
Hi, I'm using 'spring-boot-starter-data-r2dbc' version: '2.3.9.RELEASE' with 'r2dbc-postgresql', version: '0.8.7.RELEASE'. After a few select queries it gets stuck and I don't receive any response anymore, there is no error. Has anybody else seen this issues? Thank you!
Ray-Neyra
@Ray-Neyra
Hello,
Where "tor-tcp-epoll" is configured? How number of available threads is calculated and is there way to manipulate it?
Ray-Neyra
@Ray-Neyra
I honestly dont understand why locally I have "or-tcp-epoll" and "tor-tcp-epoll" and when in reqeust spring r2dbc code is executed it switch from first to second thread pool and on env I have just "reactor-http-epoll" and everything is executed on it.
TH Lim
@sshark
hi. I have set the property spring.r2dbc.url=r2dbc:pool:mysql://localhost:3306/foodb with spring.r2dbc.pool.initial-size=5 and spring.r2dbc.pool.max-size=10. How can i verify this is the right configuration thru debug log? thanks
Richard Kollcaku
@ricardkollcaku

Hi
i am using r2dbc with maria db and it works well. When i try to add BlockHound.install(); in main
i get the error :
Any idea how to fix it
021-04-21 12:34:39.593 ERROR 7175 --- [actor-tcp-nio-2] o.m.r.message.flow.AuthenticationFlow : Authentication failed

io.r2dbc.spi.R2dbcNonTransientResourceException: Connection unexpectedly closed
at org.mariadb.r2dbc.client.ClientBase.closedServlet(ClientBase.java:248) ~[r2dbc-mariadb-1.0.0.jar:1.0.0]
at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:250) ~[reactor-core-3.4.5.jar:3.4.5]
at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:259) ~[reactor-core-3.4.5.jar:3.4.5]
at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:142) ~[reactor-core-3.4.5.jar:3.4.5]
at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:401) ~[reactor-netty-core-1.0.6.jar:1.0.6]
at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:416) ~[reactor-netty-core-1.0.6.jar:1.0.6]
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:470) ~[reactor-netty-core-1.0.6.jar:1.0.6]
at reactor.netty.channel.ChannelOperations.onInboundClose(ChannelOperations.java:431) ~[reactor-netty-core-1.0.6.jar:1.0.6]
at reactor.netty.channel.ChannelOperationsHandler.channelInactive(ChannelOperationsHandler.java:74) ~[reactor-netty-core-1.0.6.jar:1.0.6]

Mark Paluch
@mp911de
Please post the full stack trace showing the origin of BlockingOperationError
Richard Kollcaku
@ricardkollcaku
this is all
Arun Kumar Balasubramani
@arunkumarpro_gitlab
Hi, it looks like the Json wrapper of r2dbc-postgresql driver does native JSONB encoding which is not preserving the order of the keys while converting from String. For my use case, I would like to preserve the order of the JSON keys and store it in the JSON typed column. Is there any work-around to do this?
Petromir Dzhunev
@petromir

@mp911de I think I experience connection leak similar to what is decribed here r2dbc/r2dbc-pool#54.

Background

I've run locally performance tests which do ~25 requests/second using local PostgreSQL server. After finishing of the tests I observe connection leak.
Same happens when tests are executed against production environment, but we use PgBouncer there, which also shows a connection leak

Observation for a single application instance running locally

These are our connection pool settings

CONNECTION_POOL_INITIAL_SIZE=0
CONNECTION_POOL_MAX_SIZE=21
CONNECTION_POOL_MAX_IDLE_TIME=1000
CONNECTION_POOL_MAX_CREATE_CONNECTION_TIME=10000
CONNECTION_POOL_ACQUIRE_RETRY=3
CONNECTION_POOL_MAX_ACQUIRE_TIME=7000

Metrics exposed to Prometheus

r2dbc_pool_allocated_connections{name="connectionFactory",} 4.0
r2dbc_pool_idle_connections{name="connectionFactory",} 4.0
r2dbc_pool_acquired_connections{name="connectionFactory",} 0.0
r2dbc_pool_max_allocated_connections{name="connectionFactory",} 21.0
r2dbc_pool_pending_connections{name="connectionFactory",} 0.0
r2dbc_pool_max_pending_connections{name="connectionFactory",} 2.147483647E9

I get from JMX (which is same a Prometheus)

AcquiredSize  0
AllocatedSize 4
IdleSize 4
MaxAllocatedSize 21
MaxPendingAcquireSize 2147483647      
PendingAcquireSize 0

I get from PostgreSQL server

active_conn | idle_conn
------------------------+----------------------
                      0 |                   21
(1 row)

Versions:

org.springframework.boot - 2.2.8.RELEASE
r2dbc-pool - 0.9.0.M1
Mark Paluch
@mp911de
Can you provide a reproducer so that we can diagnose the issue?
Petromir Dzhunev
@petromir
@mp911de Any other information that I can provide, as the project can not be shared?
Mark Paluch
@mp911de
We cannot then investigate and debug the actual state of connections and the pool.
Jonathan Rodriguez
@jarpz
Hi, I wanted to know if there is support for Azure Always Encrypted feature (https://docs.microsoft.com/en-us/sql/connect/jdbc/using-always-encrypted-with-the-jdbc-driver?view=sql-server-ver15) using R2dbc driver?, we see documentation using it with jdbc. Is there any plan to supported?
Regards
Mark Paluch
@mp911de
@jarpz The SQL Server driver was built by the open source community as proof of concept for R2DBC with SQL Server. It is by no means aligned feature-wise with Microsoft's JDBC driver. You might want to reach out to your Microsoft sales rep so Microsoft can get involved into R2DBC driver development so you can use commercial features.
Petromir Dzhunev
@petromir
@mp911de I've opened a new issue r2dbc/r2dbc-pool#115. It has a link to example repository.
TH Lim
@sshark
I have a question about the reconnection in r2dbc-pool and Hikari pool. When the pool connection is left idle longer than what is set in the db server, the server will kill these connections. In my setup. for Hikari pool, it would reconnect. For r2dbc, it reconnected and failed because there were "too many connections". How does Hikari able to maintain the broken connection and r2dbc-pool can't? Btw, I have set the idle-time to 2s in r2dbc-pool to work around broken connections issue. These were some of the suggestions I read from the issue tracker
Mark Paluch
@mp911de
In R2DBC Pool, connections do not get reconnected in a reconnecting sense, rather the pool throws away the connection and attempts to create a new one.
Jonathan Rodriguez
@jarpz

@jarpz The SQL Server driver was built by the open source community as proof of concept for R2DBC with SQL Server. It is by no means aligned feature-wise with Microsoft's JDBC driver. You might want to reach out to your Microsoft sales rep so Microsoft can get involved into R2DBC driver development so you can use commercial features.

Thanks.

TH Lim
@sshark
@ricardkollcaku without the full stack trace I can’t say for sure what’s the problem. What I had encountered before has something to do with the ssl context where the driver loaded the certifications using Java.IO which is blocking
Petromir Dzhunev
@petromir
@mp911de I would like to contribute to r2dbc/r2dbc-pool#115. Should I be added as a maintainer, or I need to do a fork of the repository.
This https://github.com/r2dbc/.github/blob/main/CONTRIBUTING.adoc#submit-a-pull-request also states that You must have the right to submit code changes. Is it only required to add Signed-off-by in commit message or I should do something in addition?
Mark Paluch
@mp911de
Please fork the repo and submit a PR @petromir. You must have the right to submit code changes is a legal requirement that basically means that you must be authorized to submit code. If you write the code on your own, then likely it's your own decision whether you authorize yourself. Moreover, if you write code on behalf of your company/employer/… and the code is owned by someone else, then you need permission from them to submit code changes. By adding Signed-off-by you express that all of that is given. You can read more about DCO at https://en.wikipedia.org/wiki/Developer_Certificate_of_Origin
Petromir Dzhunev
@petromir
Got you. How to make sure that somebody else is not working on this, so we don't repeat our work?
Mark Paluch
@mp911de
Leaving a comment on the ticket works typically for letting folks know that someone is already on it.
Vadim Bryksin
@Bryksin
Guys, what is the best approach so far for managing schema and tables with R2DBC?
How to init empty DB automatically at the start of Spring Boot application, or update tables if a model was changed?
ashish-kumar-soni
@ashish-kumar-soni

Hello Guys, I am trying to read composite data type from R2DBC driver-
My table looks like below
CREATE TABLE test_udt
(
id uuid,
changes change_request_changes
)
CREATE TYPE change_request_changes AS
(
field_name text,
old_value text,
new_value text
);
I am trying to read the above table data using below code but it fails for decoding error
public Mono<TestUdt> findAll() {
return client.sql("select * from test_udt").map(row -> {
Change change = new Change(row.get("(changes).field_name", String.class), row.get("(changes).old_value", String.class), row.get("(changes).new_value", String.class));
return new TestUdt(row.get("id", UUID.class), null);
}
).first();
}

Please help me understand this. Thanks

Mark Paluch
@mp911de
What database system are you using, @ashish-kumar-soni ?
4 replies
Timur Shaidullin
@timur-sh

Hello!

I try to insert a list of items to PostgreSQL database via R2DBC, but it does not complete. Configurations with example are below.

Could anyone get an advise why this code does not hit a database?

 private Flux<OnlineChartItem> batchInsert() {
        return Mono.from(chartInputConnectionFactory.create())
                .log("start#1")
                .flatMapMany(connection -> {
                    Statement statement = connection.createStatement(INSERT_INTO_ONLINE_CHARTS);
                    for (OnlineChartItem item : items) {
                        this.prepareInsertStatement(statement, item).add();
                    }
                    return Flux.from(statement.execute());
                })
                .thenMany(Flux.fromIterable(items));
    }

    @Bean
    public ConnectionFactory chartInputConnectionFactory(R2dbcProperties r2dbcProperties) {
        ConnectionFactoryOptions factoryOptions = ConnectionFactoryOptions.parse("r2dbc:pool:postgresql://localhost:5432/my_db")
                .mutate()
                .option(ConnectionFactoryOptions.USER, r2dbcProperties.getUsername())
                .option(ConnectionFactoryOptions.PASSWORD, r2dbcProperties.getPassword())
                .build();
        ConnectionFactory connectionFactory = ConnectionFactories.get(factoryOptions);

        R2dbcProperties.Pool pool = r2dbcProperties.getPool();
        ConnectionPoolConfiguration configuration = ConnectionPoolConfiguration.builder()
                .connectionFactory(connectionFactory)
                .maxSize(20)
                .initialSize(10)
                .acquireRetry(Integer.MAX_VALUE)
                .maxAcquireTime(Duration.ofSeconds(2))
                .validationQuery("select 1")
                .build();
        return new ConnectionPool(configuration);
    }
yapee123456
@yapee123456
hello
Vadim Bryksin
@Bryksin

Hi everyone, im suddenly start getting mysql connection exceptions some how related to SSL:

dev.miku.r2dbc.mysql.client.MySqlConnectionClosedException: Connection unexpectedly closed
********
javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)

Any idea how to fix it ?

Mark Paluch
@mp911de
Either downgrade your Java version to an older one that doesn't disable TLS 1.0/1.1, upgrade to MySQL 8 or wait for this change: mirromutth/r2dbc-mysql#182
Charles Harvey III
@charlie-harvey
@Bryksin Just add a bunch of "turn off SSL" stuff to your connection string
jdbc:mysql://my-host:3306/myDb?sslMode=DISABLED
There are more flags but they are all deprecated and this one should work.