Ported in a few new testcases t…More unicode, bytestring fixes.…(compare)
Dec 25 2018 12:35
coveralls commented #300
Dec 25 2018 12:35
coveralls commented #300
It keeps track of all the calls bound for each database connection and then fires them off in different threads
The keyspace Hash objects also allow you to specify the encoding of the fields, and serialize it as json automatically, or convert to ints or floats, etc.
All of these concepts are borrowed from what I used to build marcopolo.me over the past few years. This is just a better version of it, because I got to re-architect it now that I know all the things I needed to build.
let me know if you have any questions. I am going to be giving a talk on all of this at redisconf at the end of the month. So I need to get better at explaining it. :-)
ah cool that you are doing a talk about it, i hope videos is posted from them so i can watch later :D
i have not seen that many talks that even touches redis clustering so far
Hi by using this ( https://github.com/Grokzen/docker-redis-cluster ) this code base and composing it to create redis cluster on docker, after adding some line related to PIP command in the Docker file it got executed. but Iam not able to connect with all the masters its only allow me to connect with 7002 port. I am using redis cluster for Session management in MVC with C#, it throws below exception when I changes the port in my config file it throws error- MOVED, But when I pass all the IP:PORT in the connection string it works and when 7002 port is removed from the list it again throws error ( have used RedisSessionStateStoreProvider) and for Microsoft.Web.Redis.RedisSessionStateProvider gives error for connection Multiplexer. it always says to increase the time, which does not work. Please suggest.
@santosh2812 Hi. sorry for not responding sooner
First look if you are using the ports 7000-7007 to other services that prevents you from reserving them for redis cluster inside docker. The second you can check is if when you start the cluster that docker binds all ports for you or not on your host system. If not then you have to debug why that happens.
The other things i can comment on is that are you really using a cluster compatible client? The client must be capable of understanding the redis cluster spec and commands. You can't really use any old regular redis client and expect it to work in a cluster env
are you able to connect to all nodes in the docker cluster with the regular redis-cli client from outside the container?
enter your container and look if all redis-server processes is running and if they are using the correct ports
On the node that works, what do you get back when you run CLUSTER INFO command?
I do think that you are using a redis client that is not compatible with a redis cluster and you should look into using a client that is for your usecase
@Grokzen Hi, its ok. as the setup was on single container so and also was not able to figure out the problem So i started Sentinel configuration. but here also am not able to configure and understand the configuration I have some points please have a look. As per the tutorial (https://github.com/AliyunContainerService/redis-cluster )the configuration is created with local IP of docker(172.17.0.2) when I do port forwarding in Docker file for the master and sentinel its available for connecting from local machine but the synchronization between the master and slave did not happen it throw below exception Connecting to MASTER 192.168.80.169:6379 1:S 06 Jun 06:24:12.792 MASTER <-> SLAVE sync started 1:S 06 Jun 06:24:12.800 Non blocking connect for SYNC fired the event. 1:S 06 Jun 06:24:12.862 Master replied to PING, replication can continue... 1:S 06 Jun 06:24:12.869 Partial resynchronization not possible (no cached master) 1:S 06 Jun 06:24:12.870 * Master does not support PSYNC or is in error state (reply: -ERR Can't SYNC while not connected with my master). Again to get the Master information we need to connect with Sentinel so it needs to be on port forwarding, when I specify the port forwarding on it then unable to scale it as we need more than 1 sentinel for electing the master in case of master failover. Here below exception is thrown ERROR: for redissentinel_sentinel_3 Cannot start service sentinel: driver failed programming external connectivity on endpoint redissentinel_sentinel_3 (91803d790d07d933f9bfc7c2c431b23f8d633db37f6981705e1f758b07080c6a): Bind for 0.0.0.0:26379 failed: port is already allocated Apart from above blockers there are lots of question coming as below which I have also posted on the forum but did not got any response on it, some of them are :
How to expose the port 26379 by using sentinel announce-ip (my system Ip) to connect from outside.
As we will get the master Information from The sentinels - which one to connect as I have 3.
Do we need to right the logic to connect with the master in our application (MVC C#) to connect with the Details got form the point no 2. Or How.
Do we need to expose the Slaves also? How.
@santosh2812 Sentinel? I can't help you with sentinel setup, i have never used that setup. I have only used the built-in redis-cluster support inside redis. You probably have to turn to other more general redis channels to get help with sentinel setups
Another more general problem you appear to have is that you have problems with your port sharing from the docker container. There is not many things that can cause that
one main thing is that you either run multiple containers that expose the same port, run one container but try to expose the same port multiple times, or that you have redis installed/configured on your host machine and try to start a container with the same setup and they both want to use the sam port
the error port is already allocated is usually a indication or either point 1 or 3 from ^
This PR andymccurdy/redis-py#866 at redis-py might have some interesting implications for this cluster impl if merged
seems to make fairly big changes to the client methods, by adding support for namespacing functions. I haven't had a play with the code yet but this might be useful for auto hashtagging keys (hashtag as in https://redis.io/topics/cluster-spec#keys-hash-tags)
@AngusP Is that not the third/fourth time that a namespace MR has been created? O.o
but still no merge yet
@Grokzen Re: that earlier work I was doing to find a good way to support lazy initialization, we were able to solve it at the application layer.
We have a delegate object we use via something like this: def getattr(self, attr): if not self._redis: self._redis = initializeRedisClient(self._clusterEnabled, self._config) res = getattr(self._redis, attr) return res
works a lot better than expected
without needing to touch the library code at all
I'm now looking to add something like your https://github.com/Grokzen/docker-redis-cluster docker container to our testing setup. Would it be possible for you to add on an MIT license to that repro? I'd love to use your work directly, but it always nice to be sure there is a license explicit.
@AlexBoyd Woot? O.o is there not MIT license for that repo already?
It should have been that added to the repo already
i will add
awesome. Thanks for that :)
Grokzen/docker-redis-cluster#25 PR to add the default MIT license if that would at all help.
Hi there! I've been using the redis-py-cluster module for a few days now, and I've run into an issue. For rc.cluster_nodes(), I run into an issue where my port is returned as 6379@16379, and it errors out because in the utils.py, def parse_cluster_nodes(...) uses 'port': int(port). I've changed this to str(port) which hasn't caused any issues in my setup, but I'm not sure if it would cause issues elsewhere.
is there a good reason for that? it should have the same restriction as the non-pipelined mode and only works with keys mapped to the same hashslot but I'm only planning on using single-key calls anyway.
Put up a small patch to fix a bug I found in the retry logic when hitting a timeout / connection error in pipelines
just a really small fix where a return was inside rather than after a loop.
I had install redis-python-cluster model on AWS VM, then test the redis cluster connect failed. So is that only support redis 3.0+ version?
But I use jedis test it was successfully
Hello everyone! I am new to redis-py-cluster . I run a rediscluster with redis4.0.1, redis-py-cluster proper functioning. But redis-py-cluster's document said that It only support redis 3.0.x. I am confused.
And What redis version redis-py-cluster can realy supported? the redis-py-cluster i am using is redis_py_cluster-1.3.4.
morning guys! i anybody can recommend async/await style rediscluster client for py3.6 what better to use with aiohttp / sanic or any async io http servers ?
hello, how about let pipeline.watch() and pipeline.multi() support unique slot operation? Is there a good method to solve this problem before watch() and multi() available?
@alwaysR9 There is not that i know of
I'm getting an error AttributeError: 'ConnectionPool' object has no attribute 'nodes'