Lettuce/Lobby

Chat room for the Lettuce Redis driver project

Mark Paluch

@mp911de

@pendula95 That's a bug. can you submit a bug report including the stack trace?

Grigoriy Skobelev

@GSkoba

Hi all, I started write program with redis 6.0.1 and chose lettuce as a client) At now I have some question, please help me.
Redis 6.x arrive with multithreading for faster I/O - lettuce provide performances impact from this feature (https://github.com/lettuce-io/lettuce-core/issues/360)? When we use connection we can keep open connection or we need for open/close connection in all operations?

Mark Paluch

@mp911de

Lettuce isn't necessarily faster, it just deals differently with available resources. You can keep a single connection open unless you use blocking operations (BLPOP) or Redis transactions across threads.

Tokuhiro Matsuno

@tokuhirom

Is there a maintenance plan for older version of lettuce? I want a page like this: https://github.com/spring-projects/spring-framework/wiki/Spring-Framework-Versions#supported-versions

ah, i found it https://github.com/lettuce-io/lettuce-core/wiki/Lettuce-Versions

eladyac

@eladyac

Hi, I run stress test which I try to set the same key multiple times simultaneously. The API does not return any errors, but the key is not set on Redis. What can be the problem?

Mark Paluch

@mp911de

Do you have a bit of code or a debug log handy?

Myungjae Won

@mjaewon

Hi all, does Lettuce provide any API to return unhealthy redis shards in a redis cluster?

Myungjae Won

@mjaewon

A bit of background here - Thinking of implementing a shard-level circuitbreaker, which fails fast and fallbacks to something else (i.g. calling database), when a redis shard becomes unhealthy.

Mark Paluch

@mp911de

You can obtain the node flags via Partitions (e.g. fail, eventual fail).

Beyond that, you can obtain connections to each node and check the isOpen method

Myungjae Won

@mjaewon

Thank you. That would work. One follow-up question - Is there any way with Lettuce to get updated when redis cluster's topology changes (i.g. a redis shard becomes unhealthy) ? Or do I need to retrieve each node's status periodically by myself?

Mark Paluch

@mp911de

Lettuce performs a refresh itself. You can listen to topology refreshed events (ClusterTopologyChangedEvent) that are published whenever the topology gets refreshed

Myungjae Won

@mjaewon

Great. Much appreciate it

Grigoriy Skobelev

@GSkoba

Hi all, when we using tls and make connection to sentinel - its normal that lettuce use schema "redis+tls"?

pranayRapido

@pranayRapido

Hello People..
I am facing one issue in my Redis connectivity
I am using lettuce 5.1.2 version in my project.. Here the issue is
when I am connected to Redis and for some connectivity issue, it got disconnected... I can see my Redis client was able to make the connection again which means it is reconnected..
Here when I send the ping command it times out.... but read command is working fine

ClusterClientOptions clusterClientOptions = ClusterClientOptions
                .builder()
                .pingBeforeActivateConnection(true)
                .autoReconnect(true)
                .build();

These are the client option I am using... Can someone please help

pranayRapido

@pranayRapido

@mp911de ^^

southrivers

@southrivers

when i use lettuce to connect to redis cluster , i got error like this

java.lang.ClassCastException: io.netty.channel.epoll.EpollEventLoopGroup cannot be cast to io.netty.channel.EventLoopGroup
        at io.lettuce.core.resource.DefaultEventLoopGroupProvider.getOrCreate(DefaultEventLoopGroupProvider.java:137)
        at io.lettuce.core.resource.DefaultEventLoopGroupProvider.allocate(DefaultEventLoopGroupProvider.java:83)
        at io.lettuce.core.AbstractRedisClient.getEventLoopGroup(AbstractRedisClient.java:250)
        at io.lettuce.core.AbstractRedisClient.channelType(AbstractRedisClient.java:236)
        at io.lettuce.core.cluster.RedisClusterClient.createConnectionBuilder(RedisClusterClient.java:795)
        at io.lettuce.core.cluster.RedisClusterClient.connectStatefulAsync(RedisClusterClient.java:764)
        at io.lettuce.core.cluster.RedisClusterClient.connectToNodeAsync(RedisClusterClient.java:535)
        at io.lettuce.core.cluster.RedisClusterClient$NodeConnectionFactoryImpl.connectToNodeAsync(RedisClusterClient.java:1183)
        at io.lettuce.core.cluster.topology.DefaultClusterTopologyRefresh.openConnections(DefaultClusterTopologyRefresh.java:302)
        at io.lettuce.core.cluster.topology.DefaultClusterTopologyRefresh.loadViews(DefaultClusterTopologyRefresh.java:81)
        at io.lettuce.core.cluster.RedisClusterClient.fetchPartitions(RedisClusterClient.java:936)
        at io.lettuce.core.cluster.RedisClusterClient.loadPartitionsAsync(RedisClusterClient.java:905)
        at io.lettuce.core.cluster.RedisClusterClient.initializePartitions(RedisClusterClient.java:860)
        at io.lettuce.core.cluster.RedisClusterClient.assertInitialPartitions(RedisClusterClient.java:865)
        at io.lettuce.core.cluster.RedisClusterClient.connect(RedisClusterClient.java:387)
        at io.lettuce.core.cluster.RedisClusterClient.connect(RedisClusterClient.java:364)

any suggest?

Mark Paluch

@mp911de

Can you file a bug report @southrivers ?

southrivers

@southrivers

the program run successful in windows os, but failed in centos, and the error is as shown above

RedisClusterClient redisClient = RedisClusterClient.create(uris);
                StatefulRedisClusterConnection<String, String> connection = redisClient.connect();

the error happens when i get an connection

vijjukumar

@vijjukumar

I have configured LettuceConnectionFactory using RedisStandaloneConfiguration and it's working. I want to print the connection pool status details such as AverageConnectionWait,Active Connections,etc., I am unable to find the details plz help me on the same. thanks

vijjukumar

@vijjukumar

@joshfix :I am using spring redis template and lettuce client. how to print the lettuceConnectionFactory status

turbolag

@RahulKushwaha

I am trying to create a Lettuce Command which runs a LUA script, but getting ERR value is not an integer or out of range all the time. Any ideas?

pranayRapido

@pranayRapido

@mp911de can you please answer my question as well

Weichen Liu

@weicliu

Hey guys, what is the normal latency for getting 100 items using mget? I'm getting ~700ms with very low call volume and it shoots up to 25,000ms with high volume. I feel that there must be something wrong. I'm using ElastCache with cluster mode and has 8 nodes. I got the metrics from CommandLatencyEvent. Is there a way to monitor Netty, like the thread pool using Lettuce? Any idea as how to troubleshoot is appreciated!

Mark Paluch

@mp911de

It depends a lot on the actual payload size. 100 items x 10 bytes is something different than 100 x 1mb

Also, the codec may require computation time (e.g. Jackson or JDK serialization are expensive)

Weichen Liu

@weicliu

@mp911de thanks for your insights! I did check the profiler and it seems to confirm your points -- the decompression, deserialization logic is chained to RedisFuture and taking a long time. Changing to thenApplyAsync helps a lot. I'll also checkout the codec. Our payload size is 1kb. Do you think the average latency around 1000ms is expected for 100 items x 1kb of mget, or it can be reduced?

pramodsimikeri

@pramodsimikeri

Hey guys, we are using Azure Cache for Redis and using lettuce for connecting to it. The connection just works fine. However, sometimes azure does patching on the redis, during this time one of the redis instance goes down and another instance becomes primary(2 nodes). I see com.lambdaworks.redis.RedisCommandTimeoutException: Command timed out after 60 SECONDS errors until both the nodes are back up. Any way to fix this issue ?

Daniyal Maniar

@DanMan259

What's the best approach to serialize Jackson JSON objects into Redis without Spring data Redis (Had conflicts with another Spring Data CRUD repo). Having trouble finding documentation on this.

Daniyal Maniar

@DanMan259

I saw this StackOverflow post that @mp911de answered, but the link in option 1 seems out of date.

Mark Paluch

@mp911de

The first option (a RedisCodec) is always subject to run on the I/O thread and any costly serialization directly impacts performance for all users of the connection. Since connections are essentially single-threaded, you may see performance degradation. Something in front of the API call typically runs on a different thread.

Daniyal Maniar

@DanMan259

That makes sense, but does spring data Redis handle this by doing the serialization and deserialization in front of the API to avoid performance degradation. If so is there a resource where I can implement something like that on my own (The CRUD repo conflicts).

Also if you are using a Connection pool and you do one operation per connection would running the serialization operation on the connection thread still cause performance degradation.

Stepan

@TsybulskyStepan

Hey @mp911de ,
I've made some research about the issue that had place some time ago. It's all about a PING command to be send over Pub/Sub channel. I believe you remember the details so I won't dive deep here and just put a reference to your answer saying that Redis 6 and new protocol RESP3 will solve this issue. The question now is whether you have plans to implement it? We can't rely on keepalive option for now and need to go with a PING command to be sent in the channel.
Thank you for the answer!

Mark Paluch

@mp911de

@TsybulskyStepan using Redis 6 with Lettuce 6 will no longer cause the risk of considering a PING response as Pub/Sub message as Pub/Sub messages are now properly identified in RESP3

Stepan

@TsybulskyStepan

Yeah, it's true, but when we use Lettuce 6 we're getting

io.lettuce.core.RedisException: Command PING not allowed while subscribed. Allowed commands are: [PSUBSCRIBE, QUIT, PUNSUBSCRIBE, SUBSCRIBE, UNSUBSCRIBE]

as Lettuce prevents us from sending PING. Your commit that checks it is still in the master codebase.

Mark Paluch

@mp911de

Good catch. I think we can remove that guard entirely for Lettuce 6.0.x and above

Stepan

@TsybulskyStepan

That would be great! Thanks

Mark Paluch

@mp911de

Can you file a ticket (or submit even a pull request, if you want)?

Stepan

@TsybulskyStepan

I'll create a ticket for now. If I find time then I will also prepare a pull request. :)

Francis

@Ciccios

Hello, I am using Lettuce for the first time on a project using redis stream. I have read the documentation and played a bit with the client but I cannot find a way to set and forget a redis stream reader that waits until data is available and push it back to the client. When I execute an xreadGroup command it returns immediately an empty set. I thought it was kind of registering an asynchronous thread which is woken up when data was available in netty but in reality it is not. What am I missing? Could someone help me on this? I would really be grateful clarification on this. Thank ou

Mark Paluch

@mp911de

XREAD has two operating modes: Immediate return (default) and blocking (using the BLOCK option). See https://redis.io/commands/xread. There's no such thing as pushing stream messages because that's not how Redis streams work. It's pull-oriented

If you're looking for an inversion of this principle, you need a component that regularly polls the stream and notifies your application. Have a look at Spring Data Redis with its message listener containers, see https://docs.spring.io/spring-data/redis/docs/current/reference/html/#redis.streams.receive.containers.

Francis

@Ciccios

@mp911de I am aware that redis commands can be sent in clear via socket connection and so as such I don't think it is about how redis stream work nor that it is pull oriented only(Actually I think is the opposite) but how the client side is implemented. I am telling this because I have tried it by creating a NIO non blocking java socket connection to redis port 6379 and sent the xreadgroup command with BLOCKING MODE (Socket Write) and registered a READ interest on the selector. So when I pushed a message in the stream the selector on my side asynchronously received the message without having a separate thread on active socket read0 wait. I think this should be the behaviour for streams in lettuce to fully benefit from Netty async IO and to gain the best efficience. Spring Redis uses the poll mechanism which is still very inefficient ... so unless I am missing something else Lettuce could be improved on Redis Stream
At the current state I cannot use Lettuce for my purpose as I would need to have thousands of consumers waiting for messages and the polling mechanism would be overkill. Let me know what do you think about this. I am happy to discuss

Mark Paluch

@mp911de

Thanks for clarifying. Initially, your question sounded a bit different. The entire I/O side, selector operations (NIO) or native transports (kqueue, epoll) are fully encapsulated by netty and our interface is the Channel respective its pipeline. All threading (also, binding the I/O thread to the channel) is netty business where we don't interfere with.

Lettuce uses a future-based API to notify its caller once the response has arrived (or the command has timed out) so I'm not sure there's much we can do here. Also, each stream reader that wants to use blocking read mode, has either to await the next read as the Redis protocol isn't multiplexed or use an individual socket for XREAD

Where communities thrive

People

Repo info

Activity