Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 02:38
    ttddyy closed #71
  • 02:38
    ttddyy commented #71
  • 02:33
    ttddyy edited #78
  • 02:31
    ttddyy commented #78
  • 02:29
    ttddyy closed #78
  • 02:29

    ttddyy on main

    Use caller's classloader when c… (compare)

  • 02:29

    ttddyy on 0.8.x

    Use caller's classloader when c… (compare)

  • Dec 04 23:48
    jaswanthbellam edited #360
  • Dec 04 23:02
    jaswanthbellam edited #360
  • Dec 04 20:47

    mp911de on main

    Remove travis build file as we … (compare)

  • Dec 04 20:47
    mp911de closed #359
  • Dec 04 18:24
    jaswanthbellam edited #360
  • Dec 04 18:22
    jaswanthbellam opened #360
  • Dec 04 16:46
    davecramer opened #359
  • Dec 04 12:38
    petromir opened #78
  • Dec 04 11:14
    mp911de edited #159
  • Dec 04 11:14
    mp911de edited #170
  • Dec 04 11:14
    mp911de edited #170
  • Dec 04 07:15
    mp911de commented #222
  • Dec 03 22:27
    jaswanthbellam commented #222
Dan Cohen-Smith
@dancohensmith
and yeah your right we need a benchmark
Ivan Kushmantsev
@climber73
Why does not r2dbc-postgresql support 40001 error code (serialization_failure)?
Currently, when db returns 40001, io.r2dbc.postgresql.ExceptionFactory puts this error in io.r2dbc.postgresql.PostgresqlTransientException
and then in org.springframework.r2dbc.connection.ConnectionFactoryUtils it is wrapped in org.springframework.r2dbc.UncategorizedR2dbcException.
Mark Paluch
@mp911de
@climber73 is that a driver or a Spring issue?
Ivan Kushmantsev
@climber73
As far as I can understand this patch fix the issue. So it's driver.
Mark Paluch
@mp911de
Thanks for clarifying, it wasn't clear from the description above. Do you want to submit the patch as pull request?
Ivan Kushmantsev
@climber73
@mp911de take a look please: pgjdbc/r2dbc-postgresql#355
Mister_Jesus
@Bittuw
Hello, why MappingR2dbcConverter use Row::get(String) instead Row::get(String,Class) when reading values?
Mark Paluch
@mp911de
That's a Spring Data question.
Mister_Jesus
@Bittuw
@mp911de , why? Its R2DBC implementation
Mark Paluch
@mp911de
MappingR2dbcConverter is part of Spring Data R2DBC. Spring Data does not implement R2DBC, it uses R2DBC.
Mister_Jesus
@Bittuw
@mp911de , oh, my mistake. I seen R2DBC in name and opened gitter :)
Koizumi85
@Koizumi85

Hi everyone. We are currently facing a strange issue were queries are not executed
Details: We have a reactive web application which is secured using spring security and so we wrote a ReactiveUserDetailsService which loads our user using Spring-R2DBC Repositories in two steps:

  1. Load the user
  2. Load the authorities if the user was found in the database
    Code looks something like this:
    @Override
    public Mono<UserDetails> findByUsername(final String username) {
     final Mono<UserProjection> userMono = userRepository.findProjectionByUsername(username);
     return userMono
             .flatMap(u -> {
                 return authorityRepository.findByUsername(u.getUsername())
                     .map(a -> new SimpleGrantedAuthority(a.getAuthority()))
                     .collectList()
                     .flatMap(l ->
                     Mono.just(
                         User.withUsername(u.getUsername()).password(u.getPassword())
                         .authorities(l).disabled(!u.isActive()).build()));
             });
     }

Now sometimes it happens that this method is executed (a breakpoint in there is hit), but there is not a single Query executed (according to the logs with logging.level.org.springframework.r2dbc = DEBUG)
So even the userRepository.findProjectionByUsername() call doesn't do anything

Anyone an idea what happens there?

Igor Šušić
@igor-susic

@mp911de Regarding the issue:

My thoughts are that command COPY TO STDOUT should be suported the same way the DataRow is suported in src/main/java/io/r2dbc/postgresql/PostgresqlResult.java because it is result and official documentations says that:

The backend sends a CopyOutResponse message to the frontend, followed by zero or more CopyData messages (always one per row), followed by CopyDone.

Which tells me that PostgresqlRow::toRow should maybe be overloaded for CopyDatatype of message also. And as per documentation it should be expected that driver is capable of delivering the COPY TO STDOUT result to the client.

When STDIN or STDOUT is specified, data is transmitted via the connection between the client and the server.

For example:

if (message instanceof CopyData) {
        PostgresqlRow row = PostgresqlRow.toRow(this.context, (CopyData) message, this.rowDescription);
        sink.next(f.apply(row, this.metadata));
 }

Now this should probably for start be supported in simple query protocol as I don't see any obvious use for exteneded one.

For the COPY FROM STDIN I guess we should be somehwere close to client folder as per documentation we should sent data as multiple CopyData messages. Now, STDIN does not need to be real STDIN in a sense, it could be file, or java stream, some array maybe etc. so that support should go to the cliend in some sense.

Now it's just high overview of the idea, I just wan't to get your opinion, to see is it the right direction and provide more detailed overview soon. And I know I took some time, but I'm little junior to reactive Java, this driver and life happens so :D

Thanks :bow:

Mark Paluch
@mp911de
We can add another method to PostgresResult to apply a mapping function to CopyData.
We probably should not use map because it already accepts a BiFunction. Maybe we should even consider an extended API design similar to Mutiny (on().row().map(…), on().copy().map()) but that's more food for thought.
Mark Paluch
@mp911de
For the start, let's introduce a mapCopy(Function<CopyData, T>) method to play around with a few use cases before we move on. We're going to learn what looks and feels best when playing with code.
For copy in, we could probably do something on the Statement API, either withCopyIn or overload execute accepting a publisher of buffers or a CopyIn wrapper.
Anyways, let's start with copy out for now to see how it goes @igor-susic
Igor Šušić
@igor-susic
Sure, I'll do just that for the start :)
Ivan Trendafilov
@IvanGTrendafilov
Hello guys, I'm facing some issues and I would be very thankful if somebody give me a proposal or advice. When my service use postgresql pgbouncer with session pool mode everything works correctly and no issues found, but when I switch my service to use pgbouncer transaction pool mode, I'm facing behavior and errors like -> ("bind message supplies 4 parameters, but prepared statement “" requires 11; OR execute; bad SQL grammar [UPDATE translations\nSET text = $1,\n user_updated = $2,\n modified = $3\n where id = $4\n]; nested exception is io.r2dbc.postgresql.ExceptionFactory$PostgresqlBadGrammarException: [22021] invalid byte sequence for encoding \"UTF8\": 0x00\ ) May you give me some proposal or advice for. I am using latest versions of r2dbc pool and driver. Also preparedStatementCacheQueries option is set to 0. Thanks in advance!
Mark Paluch
@mp911de
This is because of statement caching
You can configure to disable statement caching via preparedStatementCacheQueries when using the URL-based configuration mechanism or the preparedStatementCacheQueries() method on the config builder @IvanGTrendafilov
9 replies
Marcos de la Calle
@marcosDLCS_twitter
Hi, I don't know if this is the correct room for my issue. Is it possible to init a 'TestContainer' Postgres database from Spring with a R2DBC connection?
Mark Paluch
@mp911de
@marcosDLCS_twitter check out Testcontainers support channels at https://www.testcontainers.org/getting_help/
William S Johnstone III
@wjohnst3
Hey, I am playing with r2dbc against a mysql db. The database requirers ssl. I have the needed pem files. Does anyone know how to config this?
springboot 2.4.0 by the way
Mark Paluch
@mp911de
@wjohnst3 have you consulted the driver documentation at https://github.com/mirromutth/r2dbc-mysql/#programmatic-configuration?
William S Johnstone III
@wjohnst3
Thanks Mark, this will help.
Ivan Kushmantsev
@climber73
Hi, guys! As I wrote before, it looks like there is a subtle issue somewhere in r2dbc-pool, r2dbc-postgresql or even in projectreactor. Here is a small project to reproduce the issue.
Please help with investigation of it. I understand that the test looks like a bit far-fetched, but it models our case we faced during development of billing application with CockroachDB as its database.
Mark Paluch
@mp911de
Thanks @climber73. Can you file a ticket against r2dbc-pool? I don't think it's a Boot issue in the first place. For some reason, your first message got lost in transit.
1 reply
Petromir Dzhunev
@petromir

Hi, thanks for the good work.
I would like to use r2dbc-proxy together with r2dbc-pool. Here is what I have so far:

        final ConnectionFactory connectionFactory = ConnectionFactories.get(ConnectionFactoryOptions.builder()
                .option(DRIVER, "pool")
                .option(PROTOCOL, "postgresql")
                .option(PORT, postgreSqlReactiveConfig.getPort())
                .option(HOST, postgreSqlReactiveConfig.getHost())
                .option(USER, postgreSqlReactiveConfig.getUsername())
                .option(PASSWORD, postgreSqlReactiveConfig.getPassword())
                .option(DATABASE, postgreSqlReactiveConfig.getDatabase())
                .option(MAX_SIZE, connectionPoolConfig.getMaxSize())
                .option(PostgreSqlReactiveConfig.PREPARED_STATEMENT_CACHE_QUERIES_OPTION, 0)
                .build());

How this can be combined with

ConnectionFactory connectionFactory = ConnectionFactories.get(ConnectionFactoryOptions.builder()
        .option(DRIVER, "proxy")
        .option(PROTOCOL, "postgresql")
        .build());
Mark Paluch
@mp911de
You should be able to specify pool:postgresql as PROTOCOL and proxy as DRIVER in your first example
R2DBC uses a delegation pattern. Using an URL like r2dbc:proxy:pool:postgresql://host:port/?maxSize=…& preparedStatementCacheQueries =… makes clear where this comes from.
Petromir Dzhunev
@petromir
Thanks, didn't know that. I will try and let you know if I experience any issues.
Petromir Dzhunev
@petromir
One more question. Should I copy paste examples at r2dbc-proxy-examples in order to use TracingExecutionListener. I couldn't see any listener included as part of r2dbc-proxy project.
Mark Paluch
@mp911de
The proxy project is an infrastructure project for now without concrete implementations. Each implementation is opinionated in some way and could require additional dependencies. We didn't want to ship these yet. Paging also @ttddyy
Petromir Dzhunev
@petromir
yeah, I've noticed that tracing uses brave, while I need opentracing, so for now copy-pasting with some adjustments:)
Mark Paluch
@mp911de
I think it makes sense for these kinds of integrations to live in their main project instead within r2dbc-proxy.
Christoph Sturm
@christophsturm
the postgresql r2dbc client logs constraint violations as a warning. I think thats wrong because i handle the constraint violation in my code.
10:14:36.179 [reactor-tcp-nio-2] WARN  io.r2dbc.postgresql.client.ReactorNettyClient - Error: SEVERITY_LOCALIZED=ERROR, SEVERITY_NON_LOCALIZED=ERROR, CODE=23505, MESSAGE=duplicate key value violates unique constraint "users_email_key", DETAIL=Key (email)=(email) already exists., SCHEMA_NAME=public, TABLE_NAME=users, CONSTRAINT_NAME=users_email_key, FILE=nbtinsert.c, LINE=656, ROUTINE=_bt_check_unique
1 reply
Dan Cohen-Smith
@dancohensmith
@mp911de qq the postgres driver doesn't cache anything does it? like the schema for example? might just be cockroach but trying to unders wierd behaviour
Mark Paluch
@mp911de
There's prepared statement caching enabled by default. Other than that, the driver doesn't care about your schema at all.
Rafi Shamim
@rafiss

hi! i have a question about r2bdc-pool which i couldn't figure out since i am not so great at reading reactive code. what happens when obtaining a connection from the pool and the connection validation fails? (for example, because the backend server was restarted.) does it try to create a new connection?

i'm looking at https://github.com/r2dbc/r2dbc-pool/blob/a58db10f548997eb1ea6074fe5d5ba3483ca8cf9/src/main/java/io/r2dbc/pool/ConnectionPool.java#L109

Mark Paluch
@mp911de
When the validation fails, then the obtained connection gets closed. A new connection attempt is being issued when acquireRetry is greater zero @rafiss
Petromir Dzhunev
@petromir

I've applied the following r2dbc-proxy settings:

        List<ProxyExecutionListener> proxyExecutionListeners = new ArrayList<>();
        proxyExecutionListeners.add(tracingExecutionListener);

        final ConnectionFactory connectionFactory = ConnectionFactories.get(ConnectionFactoryOptions.builder()
                .option(DRIVER, "proxy")
                .option(PROTOCOL, "pool:postgresql")
                .option(PORT, postgreSqlReactiveConfig.getPort())
                .option(HOST, postgreSqlReactiveConfig.getHost())
                .option(USER, postgreSqlReactiveConfig.getUsername())
                .option(PASSWORD, postgreSqlReactiveConfig.getPassword())
                .option(DATABASE, postgreSqlReactiveConfig.getDatabase())
                .option(MAX_SIZE, connectionPoolConfig.getMaxSize())
                .option(PostgreSqlReactiveConfig.PREPARED_STATEMENT_CACHE_QUERIES_OPTION, 0)
                .option(PostgreSqlReactiveConfig.PROXY_LISTENER_OPTION, proxyExecutionListeners)
                .build());

        final ConnectionPoolConfiguration configuration = ConnectionPoolConfiguration.builder(connectionFactory)
                .initialSize(connectionPoolConfig.getInitialSize())
                .maxSize(connectionPoolConfig.getMaxSize())
                .maxIdleTime(Duration.ofMillis(connectionPoolConfig.getMaxIdleTime()))
                .maxCreateConnectionTime(Duration.ofSeconds(connectionPoolConfig.getMaxCreateConnectionTime()))
                .acquireRetry(connectionPoolConfig.getAcquireRetry())
                .maxAcquireTime(Duration.ofMillis(connectionPoolConfig.getMaxAcquireTime()))
                .build();

        return new ConnectionPool(configuration);

Everything works fine when requests are done to our WebFlux controllers, but when we have a thrift request we get this:

org.springframework.dao.DataAccessResourceFailureException: Failed to obtain R2DBC Connection; nested exception is java.lang.IllegalArgumentException: interface io.r2dbc.spi.Connection is not visible from class loader

@mp911de any ideas?

Petromir Dzhunev
@petromir
And to make it more interesting only the first connection to thrift works
Petromir Dzhunev
@petromir

I'm not sure but the problem might come from

return (ConnectionFactory) Proxy.newProxyInstance(Thread.currentThread().getContextClassLoader(),
            new Class<?>[]{ConnectionFactory.class, Wrapped.class, ProxyConfigHolder.class}, invocationHandler);

based on https://stackoverflow.com/a/13379769

When I print the classLoader during http and thrift requests I see sun.misc.Launcher$AppClassLoader@18b4aac2
Mark Paluch
@mp911de
Hum, that's interesting. Thread.currentThread().getContextClassLoader() might point to a different ClassLoader. You might want to file that issue in R2DBC Proxy
The originating class' ClassLoader seems the better choice.
Petromir Dzhunev
@petromir