Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    Hi all, How to set redis password in dynomite conf?
    Jelmer Kuperus
    Hi i am looking at dynomite and it seems great but my main concern seems to be that the project does not look very active and you ca't find many resources online about it
    can someone elaborate on the status of the project? is it still used by netflix etc ?
    Sailesh Mukil
    Hi, the project is still active. There will be more active development on it continuing soon. For now, there are some bandwidth issues on being able to reply to all OSS questions, we're hoping the community will be able to fill in here. But we're having one more developer join soon and keep the project more active.
    Jelmer Kuperus
    At what kind of scale is this project being used ?
    Is this a small operation within the netflix org or is it pretty core
    Sailesh Mukil
    While I'm unable to share any numbers, it has significant usage within the org and is quite stable as a distributed HA cache.
    Jelmer Kuperus
    and as a primary storage solution?
    i am looking at it as a cassandra replacement for relatively small read heavy workloads
    as cassandra tens to be "operationally complex"
    Sailesh Mukil
    @jelmerk Certain use-cases can use it as a primary datastore depending on the guarantees required. You can vet it based on the guarantees mentioned in the wiki here:
    Has anyone used this successfully in a Ubuntu environment?
    Ubuntu 20 in particular
    I'm encountering the issue described here Netflix/dynomite#456
    No idea how to proceed but I need to proceed
    Any help?
    why dynomite do not release after 2019?
    Marie Diana Tran
    interesting question, no answer so far ?
    Jelmer Kuperus
    I guess the implied question is is the project still alive
    Liem Le Hoang Duc
    Tbh, dont get why most of redis cluster solutions stop developing/releasing new version. Cordis, Dynomite, etc...
    Jelmer Kuperus
    I’d love to use dynomite but its hard to recommend right now with so little adoption and activity
    “Nobody gets fired for picking redis enterprise”
    Marie Diana Tran
    It seems like antirez initiated redis clusters but since he "kind of" retired from Redis, I am not sure this would really move forward at least for the open source project.
    Redis Entreprise seems to market itself as the way to scale Redis but it is so expensive and the pricing for Redis Enterprise Software is way too weird (per database shard) ...
    It is also quite painful to maintain & deploy Redis Cluster day to day (node lost connection issue/ sharding issue after rejoin etc.).
    The cheap way would be to use a proxy solution for Redis like Twitter/twemproxy but the project is quite dead, since the author left Twitter for starting an AI company that got bought by Etsy.
    Plus, Twitter is now using Pelikan project and still struggling with reusing twemproxy.
    Hopefully, Dynomite is reusing some part of Twemproxy and continuing the developing. So please continue the good work !
    Ioannis Papapanagiotou
    FWIW, we still run Dynomite in production (the same as in the OSS repository) and it is in a pretty stable situation for us. We will continue using it as we see bugs we will fix them. In that sense, we have not stopped developing or releasing Dynomite, to some extent, it is just stable for our use cases. We would love for the community to pick up some steam and start developing features that they need. For more context, @smukil mentioned above that there are bandwidth issues in anwering the OSS questions.
    @Marie-Diana yes Dynomite has used the baseline Twemproxy as a proxy but has extended it to become a separate project that does active-active.
    Marie Diana Tran
    Anyone has feedbacks regarding a production deployment of dynomite on GKE ?
    Marie Diana Tran
    Anyone experiences restarting issue when using dynomite ?
    We tried running 10 dynomite instances on te same datacenter and rack (node resource 6 Go with 2 vCPU) with 20k QPS but the dynomite pod keep restarting after reaching the max queue
    Hi all. We are trying to setup dynomite to run in a docker-swarm. We notice that initial node restart few times because it feels like the DNS inside docker is some kind of slow to publish name resolution thus the dn_resolve at https://github.com/Netflix/dynomite/blob/0713e43618f74fe6fb21226287fcb3ec618bc2d3/src/dyn_util.c#L427 fails and we get the following log:
    [2020-10-28 10:06:45.542] conf_handler:1228 conf: directive "dyn_seeds" has an invalid value
    would it be possible to make this dns probe a bit more resilient by adding some kind of configurable retry logic?
    Second question .. in those cases the dynomite crashes with a segfault with the following stacktrace
    [2020-10-28 10:06:45.543] dn_stacktrace:286 [0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x10330) [0x7fc039b52330]
    [2020-10-28 10:06:45.548] dn_stacktrace:286 [1] src/dynomite(string_deinit+0x10) [0x439b19]
    [2020-10-28 10:06:45.555] dn_stacktrace:286 [2] src/dynomite() [0x40ee03]
    [2020-10-28 10:06:45.562] dn_stacktrace:286 [3] src/dynomite() [0x40f4c9]
    [2020-10-28 10:06:45.568] dn_stacktrace:286 [4] src/dynomite(conf_destroy+0x4e) [0x413b2f]
    [2020-10-28 10:06:45.575] dn_stacktrace:286 [5] src/dynomite(conf_create+0xde) [0x413ada]
    [2020-10-28 10:06:45.584] dn_stacktrace:286 [6] src/dynomite() [0x416eea]
    [2020-10-28 10:06:45.591] dn_stacktrace:286 [7] src/dynomite(core_start+0x22) [0x41702e]
    [2020-10-28 10:06:45.597] dn_stacktrace:286 [8] src/dynomite() [0x441d75]
    [2020-10-28 10:06:45.604] dn_stacktrace:286 [9] src/dynomite(main+0x137) [0x441f78]
    [2020-10-28 10:06:45.611] dn_stacktrace:286 [10] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fc039494f45]
    [2020-10-28 10:06:45.615] dn_stacktrace:286 [11] src/dynomite() [0x40a039]
    [2020-10-28 10:06:45.622] signal_handler:123 signal 11 (SIGSEGV) received, core dumping
    looks like the deinit tries to remove a string which already turned to Null I believe the name of of the server configuration: string_deinit(&cs->name);
    it would be nice to double check this and make the termination more graceful?

    Hi guys, i am facing this issue when i try to start conductor server on dynomite db with redis backend
    [ERROR] 2020-11-10 09:22:47,583 RedisMetadataDAO - refresh TaskDefs failed
    com.netflix.dyno.connectionpool.exception.FatalConnectionException: FatalConnectionException: [host=Host [hostname=, ipAddress=null, port=8102, rack: us-east-1e, datacenter: us-east-1, status: Up, hashtag=null, password=null], latency=0(0), attempts=1]redis.clients.jedis.exceptions.JedisConnectionException: Unexpected end of stream.

    My config file is very simple

    Database persistence model. Possible values are memory, redis, redis_cluster, redis_sentinel and dynomite.

    If omitted, the persistence used is memory


    memory : The data is stored in memory and lost when the server dies. Useful for testing or demo

    redis : non-Dynomite based redis instance

    redis_cluster: AWS Elasticache Redis (cluster mode enabled).See [http://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Clusters.Create.CON.RedisCluster.html]

    redis_sentinel: Redis HA with Redis Sentinel. See [https://redis.io/topics/sentinel]

    dynomite : Dynomite cluster. Use this for HA configuration.


    Dynomite Cluster details.

    format is host:port:rack separated by semicolon

    for AWS Elasticache Redis (cluster mode enabled) the format is configuration_endpoint:port:us-east-1e. The region in this case does not matter


    If you are running using dynomite, also add the following line to the property

    to set the rack/availability zone of the conductor server to be same as dynomite cluster config


    Dynomite cluster name


    Maximum connections to redis/dynomite


    Namespace for the keys stored in Dynomite/Redis


    Namespace prefix for the dyno queues


    No. of threads allocated to dyno-queues (optional)


    Non-quorum port used to connect to local redis. Used by dyno-queues.

    When using redis directly, set this to the same port as redis server.

    For Dynomite, this is 22122 by default or the local redis-server port used by Dynomite.


    I am asking here because i think it is an error in the communication between redis and dynomite, that i have started with docker following this guide http://www.dynomitedb.com/docs/dynomite/v0.5.8/quick-start/. Has anyone any idea? Thanks in advance.

    Gustavo Veloso
    Does dynomite works on Arm processors (aarch64/arm64v8)?
    Hi guys, i have deployed conductor server with dynomite (redis backend) on openshift and i have tried to use a cluster with two nodes. When i try to run a workflow sometimes some tasks stuck in status "SCHEDULED", if i run conductor server with only one dynomite node i don't have this problem. Any idea? How can i setup dynomite cluster according to you? Thanks
    Bartosz Żurkowski

    Hello guys, after two weeks of searching for a sharding solution for Redis, I finally found Dynomite. It looks very promising! I love the concept of abstracting the sharding logic into a separate layer and treating Redis as a dumb backend :)

    Does Dynomite (for Redis) provide support for scaling, i.e., can I add Dynomite/Redis nodes to the existing cluster? How to handle token management and data rebalancing? Is there a manual/automated procedure to achieve this? A link to the documentation/tutorial would be extremely helpful. Thanks!

    How do you configure the redis database number to use for a particular configuration?
    Hello, can Dynomite be used for a database other than Redis?
    Mishito Huzuka
    @marimcmurtrie Would appreciate some help setting up dynomite Manager if you ever used that ?
    Ricky Schechter
    Hi, we are seeing tons of log messages per second in our dynomite pod in our Kubernetes cluster (all internal and single node) deployment, does anyone know how to quiesce it. Oh, forgot to say that we have it configured to 10 threads and using v2.30.4. They look like this (image attached), please help! Thank you!
    Ricky Schechter
    It was the verbosity level that by default it was at ( -v11) changed it to -v1 and it is fine now.
    Abhinav Porwal
    Screenshot 2021-09-02 at 2.26.09 PM.png
    hi we have been using dynomite in our production environment and today it crashed for some reason which I will mention further. But, I checked that we don't have logs of a few days in dynomite since last 1 month. something is off and I want to understand what please find the ss below
    u can see we dont have logs of many days in between
    we have not experienced down time in those days