Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 12:25
    Joan7 commented #1250
  • 12:12
    Joan7 commented #1250
  • 12:06
    mp911de commented #1250
  • Apr 06 09:40
  • Apr 06 09:31
    Travis lettuce-io/lettuce-core (master) still failing (2621)
  • Apr 06 09:08
    mp911de closed #1261
  • Apr 06 09:08
    mp911de closed #1260
  • Apr 06 09:05

    mp911de on 5.3.x

    Fix infinite command timeout #1… Upgrade to RxJava 3.0.2 #1261 (compare)

  • Apr 06 09:05

    mp911de on master

    Fix infinite command timeout #1… Upgrade to RxJava 3.0.2 #1261 (compare)

  • Apr 06 08:31
    mp911de labeled #1261
  • Apr 06 08:31
    mp911de opened #1261
  • Apr 06 08:31
    mp911de milestoned #1261
  • Apr 06 08:19
    mp911de milestoned #1260
  • Apr 06 08:19
    mp911de labeled #1260
  • Apr 06 08:19
    mp911de opened #1260
  • Apr 06 01:57
    IAmATeaPot418 closed #1083
  • Apr 05 11:53
    asherbar commented #1038
  • Apr 05 11:07
    mp911de commented #1038
  • Apr 05 10:16
    asherbar commented #1038
  • Apr 05 10:02
    asherbar commented #1038
Mark Paluch
@mp911de
Synchronous command API objects are proxied that use the async API underneath. Check out RedisTransactionalAsyncCommands and its implementation RedisAsyncCommandsImpl.
Darshit Patel
@darshitpp

Hello all, I was wondering if someone could help me with a seemingly trivial issue that I'm unable to figure out. I have the following code:

redisTemplate.multi();
redisTemplate.opsForHash().putIfAbsent(key, hashKey, String.valueOf(value));
redisTemplate.opsForHash().get(key, hashKey);
List<Object> result = redisTemplate.exec();

The result that I get is just a boolean which is the output of the putIfAbsent(), and I never get the output of get(). Am I missing something, or is this the expected behaviour? What shall I do to get the value from the transaction?

KingOfNature
@KingOfNature
image.png
i need some help , my service use spring-boot-starter-data-redis ,spring-data-redis version:2.1.3.RELEASE, io.lettuce version:5.1.3.RELEASE ,when i start my service i can connect redis client ,but , if i dont use ,ten or more minutes later , it will throw a exception : io.lettuce.core.RedisCommandTimeoutException: Command timed out after 10 second(s) . how can i resolve it
i had setted redis configure property : timeout 10 second
Darshit Patel
@darshitpp

Ok, I managed to find a workaround by using the increment command. But for reasons unrelated to the above problem, I get the the following error:

org.springframework.data.redis.RedisSystemException: Error in execution; nested exception is io.lettuce.core.RedisCommandExecutionException: ERR EXEC without MULTI at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:54) at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:52) at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:41) at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:44) at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:42) at org.springframework.data.redis.connection.lettuce.LettuceConnection.convertLettuceAccessException(LettuceConnection.java:270) at org.springframework.data.redis.connection.lettuce.LettuceConnection.exec(LettuceConnection.java:667) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.springframework.data.redis.core.RedisConnectionUtils$ConnectionSplittingInterceptor.invoke(RedisConnectionUtils.java:381) at org.springframework.data.redis.core.RedisConnectionUtils$ConnectionSplittingInterceptor.intercept(RedisConnectionUtils.java:359)

Mark Paluch
@mp911de
You need to call the entire code block in RedisTemplate.execute to reuse the same connection across multiple calls.
Darshit Patel
@darshitpp
@mp911de yes, thank you, I figured that out. Putting the link here in case anyone comes across a similar issue: https://docs.spring.io/spring-data/data-redis/docs/current/reference/html/#tx
Laurence Gonsalves
@xenomachina
We're migrating some code from RedisCommands.key() to RedisCommands.scan(). What is the "correct" way to transform the pattern parameter of keys() (which is of type K) into the matches parameter of ScanArgs.Builder.matches (which is a String)?
Mark Paluch
@mp911de
Please check out the Redis documentation that explains the match pattern (https://redis.io/commands/scan). The keys command was part of the library from its genesis and initially we assumed to use key encoding. That turns our well when representing keys as byte array or ByteBuffer. For every other case it does not work well.
Therefore, we decided to go with String in ScanArgs
Note that there's a ScanIterator utility that simplifies the SCAN API usage by hiding multiple SCAN calls behind an Iterator API.
Laurence Gonsalves
@xenomachina
Ah, ok. That makes sense. Thanks again!
Michael Thompson
@mikey-t
If I want to connect to AWS ElastiCache in cluster mode with a "configuration endpoint", do I follow the instructions for "Redis Cluster" using a RedisClusterClient or do I use the basic RedisClient with the dnsResolver/builder and just connect to the master as in the "ConnectToElastiCacheMaster" example (I'm assuming in this case "master" would be the "configuration endpoint" listed in AWS)?
Mark Paluch
@mp911de
ElastiCache provides IIRC replication mode and cluster mode. Replication mode is a sort-of managed master/replica + failover (basically a rebuild of Redis Sentinel) and the cluster mode matches to Redis Cluster.
That being said, RedisClusterClient sounds the most appropriate way to go. In doubt, run the INFO command and check out the cluster and replication section for which one is enabled.
Michael Thompson
@mikey-t
Yeah, RedisClusterClient seemed appropriate too - just seemed strange that the getting started lettuce docs have those 2 pages about ElastiCache that both use something else and not the RedisClusterClient. Also, I still have not tracked down exactly what the AWS ElastiCache "configuration endpoint" is. Whether it's the actual host you instantiate the client with or whether you have to query the endpoint for a list of hosts. Can't even access it unless you're on the VPC and the AWS docs are super cluttered. This is the page I'm trying to understand: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/Endpoints.html
Mark Paluch
@mp911de
Which pages do you mean? Here's the documentation for the cluster client: https://github.com/lettuce-io/lettuce-core/wiki/Redis-Cluster
Michael Thompson
@mikey-t
Sorry, I didn't mean to sound like the docs are bad, they're really good. I think my problem is that I'm super new to both redis and elasticache. My coworker was saying he though the elasticache configuration endpoint was something you had to query to get a list of hosts and then instantiate the client with all the hosts, but based on that aws page I was thinking you just instantiate the client with the one configuration endpoint url. Yesterday I got access to an aws account with permissions to test stuff, so I created an EC2 instance, an elasticache redis cluster (cluster mode enabled) and used redis-cli to connect to and set/get a key and it worked against the configuration endpoint.
Mark Paluch
@mp911de
The issue with Elasticache in replication mode (non-cluster) is that AWS does a lot of trickery. There can be multiple endpoints but there only source for topology details is AWS and that isn't something that Lettuce supports. For Elasticache replication mode, you need to use either the DNS-based failover which points always to an active master or you need to use MasterReplica and provide all hosts so that Lettuce is able to determine roles of each node in the replica setup.
x0a1b
@x0a1b
I am trying to build a retry/circuiting mechanism on top of lettuce. For selective cases I want to have capability of controlling my self which node I want to read/write from and control has to be very explicit and deterministic because a failure/exception counts towards a counter that controls circuit-breaker/bulk heading logic. Give StatefulRedisClusterConnection I am able to get the RedisClusterNode for given slot, and I can even figure out list of replicas. But I am not able convert RedisClusterNodeinto *AsyncCommand interface.
Any pointers on where should I look and how can I explicitly choose to pick a node I want my commands to run on?
Mark Paluch
@mp911de
@x0a1b you can obtain a StatefulRedisConnection from StatefulRedisClusterConnection.getConnection(String nodeId). Use the NodeId from RedisClusterNode and then you get a fully operational Redis connection
x0a1b
@x0a1b
nice!
x0a1b
@x0a1b
@mp911de one thing I am noticing though is when I set REPLICA_PREFERRED and kill one of my replicas in cluster I keep getting error and failover to master never happens. Despite the fact I have cluster refresh topology configured like this:
val timeoutOptions = TimeoutOptions.builder()
                .timeoutCommands(true)
                .connectionTimeout()
                .fixedTimeout(Duration.ofMillis(500))
                .build()

        val clusterTopologyRefreshOptions = ClusterTopologyRefreshOptions.builder()
                .enableAllAdaptiveRefreshTriggers()
                .refreshTriggersReconnectAttempts(3)
                .enablePeriodicRefresh(true)
                .enablePeriodicRefresh(Duration.ofSeconds(15))
                .dynamicRefreshSources(true)
                .build()

        val connectionOptions = ClusterClientOptions.builder()
                .timeoutOptions(timeoutOptions)
                .topologyRefreshOptions(clusterTopologyRefreshOptions)
                .build()

        return RedisClusterClient.create(url).apply {
            setOptions(connectionOptions)
            setDefaultTimeout(Duration.ofSeconds(1))
        }.connect().apply {
            readFrom = ReadFrom.REPLICA_PREFERRED
        }
Mark Paluch
@mp911de
New commands should be routed to heahtly connections. Commands that were previously sent to the killed node remain on the connection until either the node comes back up or the node gets removed from the cluster config
x0a1b
@x0a1b
do I have to call connect/connectAysnc again to make sure that happens?
because I don't see that and right now I am keeping StatefulRedisClusterConnection as singleton
And connect sounds expensive operation; so I am not sure if I should be calling it all the time.
x0a1b
@x0a1b
Ok so I got it to work; tuns out sometimes enablePeriodicRefresh(Duration.ofSeconds(15)) is not reloading partitions within 15 seconds
I force called reloadPartitions
x0a1b
@x0a1b
and call connect again
It's kinda weird that for connectAsync I have to call getPartition which is a blocking API
Mark Paluch
@mp911de
Partition reload is a blocking process. We rewrote it to non-blocking for Lettuce 6.0
Tugdual Grall
@tgrall
I am implementing the auth(username, password) for Redis 6 auth/ACL, for the username, it is not clear for me if I can use a String or use another structure like CharSequence (used by the password)
What is the rule?
Mark Paluch
@mp911de
We used CharSequence for the password to be less strict about the type requirements. Effectively folks will use String as value type but Strings are subject o JVM caching. Using CharSequence allows also for non-caching implementations.
The username itself is not a secret token per-se so it can be String.
x0a1b
@x0a1b
@mp911de turns out my docker was doing something stupid keeping the port and accepting connections; it was not a problem in library
NaiveMeaning
@NaiveMeaning
Hello all, I have a problem. When I use the sccan command, the count parameter with the scanargs parameter passed in doesn't seem to work. Is there a problem with the way I use it
image.png
NaiveMeaning
@NaiveMeaning
when Max scan size is set to 1, the loop only runs once, and all elements in the key are returned
pulkitjindal-wso2
@pulkitjindal-wso2
@mp911de newbie in redis world and trying to adopt lettuce as client. While doing perf test, found high latency in the GET operations as compare to PUT operation. Both are using async api in lettuce. One thing we noticed is even if we configure command timeout to lets say 75ms, the actual latency found to be much higher in the logs, specially for GET operation. Wondering if this could be related event loop threads taking time to respond ?
Mark Paluch
@mp911de
Benchmarks require a proper fixture. Happy to continue the discussion over an actual JMH benchmark.
@NaiveMeaning The code contains a bug. Do not exit the loop in values is empty. SCAN may return without results but without necessarily completing. COUNT is a hint how many iterations Redis does on the server side when attempting to find keys. See https://redis.io/commands/scan for further details.
NaiveMeaning
@NaiveMeaning
@mp911de Thank you very much! it works!:)
ankitjhil
@ankitjhil
Hi
Need to know if lettuce now has support of publish the metrics via dropwizard or micrometer?
Adi
@advaidya

I am new to Lettuce and exploring the Lettuce library for our redis cluster setup with Master-Slave-Sentinel deployment.

I have the following deployment presently -

1] Bunch of master redis instances
2] Bunch of slave redis instances (1 slave per master)
3] Bunch of sentinels monitoring all the Master/Slave instances
4] All in Kubernetes

I need the following functionality -

1] Client to connect to sentinel for getting the cluster state
2] Client writer to partition data evenly across all masters by key to be flushed to the bunch of masters
3] Client writer should have ability to receive notifications of new masters (which were slaves earlier) when there is a failover of an existing master
& the ability to update the in memory state of the cluster dynamically.
4] Client should have the ability to pipeline the requests for better performance
5] The connections to every redis instance should support pooling
6] The ability to scale up and down the number of masters-slave nodes with the client seamlessly able to catch up with the cluster state dynamically

I would like to know whether Lettuce library can support all the things I have listed above.

ankitjhil
@ankitjhil
@mp911de Need to know if lettuce now has support of publish the metrics via dropwizard or micrometer?
Mark Paluch
@mp911de
Folks are rather building their own Micrometer integration instead of contributing a PR: lettuce-io/lettuce-core#795