by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    John Sully
    @JohnSully
    I'm not sure its something I want to directly support right now, but you can easily modify the code to get what you want

    `
    diff --git a/src/cluster.cpp b/src/cluster.cpp
    index c20b0f4c..d677c93b 100644
    --- a/src/cluster.cpp
    +++ b/src/cluster.cpp
    @@ -490,7 +490,6 @@ void clusterInit(void) {
    clusterAddNode(myself);
    saveconf = 1;
    }

    • if (saveconf) clusterSaveConfigOrDie(1);

      / We need a listening TCP port for our cluster messaging needs. /
      g_pserver->cfd_count = 0;
      `

    seems markdown messed with the formatting, anyways just delete the line:
    if (saveconf) clusterSaveConfigOrDie(1);
    in cluster.cpp
    Sam Partee
    @Spartee
    wow, that works perfectly! thanks a bunch!
    John Sully
    @JohnSully
    Also Ben is in the process of getting our discourse server setup. I think a message board will work better than chat as I'm not always able to be here to answer questions immediately. Plus it will be easier to find answers to questions there. Thanks @ddorian for the idea
    Paul Chen
    @paulmchen
    👍👍
    Heng Kuang
    @hengku
    👍
    Sam Partee
    @Spartee
    Any general tips for speeding up database intake? we have turned off active re-hashing and played around with the server threads but are still getting client closed errors around 6000 clients sending ~64mb messages over a high speed interconnect. we have the clustered database on 16 Knights Landing Nodes(192gb per node), multi-master with no replicas. we are using the redis-plus-plus C++ library for our clients. I know this seems like an odd use case but any help is appreciated.
    Sam Partee
    @Spartee
    also, we cant use replicas(as per my understanding of how they work) because we cant keep mulitple copies of keys on a node as we need all the memory.
    benschermel
    @benschermel
    Hi Guys, we now have a discourse based KeyDB Community Forum available at https://community.keydb.dev/ . This will help keep topics organized, searchable and from getting buried
    John Sully
    @JohnSully
    Thanks @benschermel!
    Jérôme Mirc
    @jmirc
    quick question. We are using a cluster of elastiCache. I did a backup of the entire cluster and I would like to what is the best way to import all RDB in keydb
    smartattack
    @smartattack
    Is there a reference to changes in rpm packages? I'm wondering what's changed betweenkeydb-5.3.3-2.el7.x86_64.rpm vs keydb-5.3.3-3.el7.x86_64.rpm, for example.
    Also, on Centos 7 and seeing strange behavior. KeyDB appears to restart randomly after many hours or many days. There is nothing in the logs in verbose mode other than restart messages, and systemctl reports: "keydb.service: main process exited, code=killed, status=6/ABRT"
    Does this imply a crash?
    John Sully
    @JohnSully
    Hi @smartattack there were two bug fixes done within 24 hours of the release. The first was issue #161 specific to RPMs and the second was a fix specific to 32-bit builds (see the trailing comments on issue #143)
    We should be tracking release notes for these updates, I apologize for that.
    smartattack
    @smartattack
    @JohnSully - thank you, that is helpful.
    John Sully
    @JohnSully
    Also for you SIGABRT crashes, this isn't a signal that triggers KeyDBs error reporting. You may need to see what's in the stderr output of the server
    Its a very unusual way to end the process I only saw it once when a version compiled for a newer distro release was used on an older release
    smartattack
    @smartattack
    I don't think there's much in stderr/stdout -> looking at journalctl -u keydb.service this is what is logged:
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com systemd[1]: keydb.service: main process exited, code=killed, status=6/ABRT
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: Usage:
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: kill [options] <pid|name> [...]
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: Options:
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -a, --all do not restrict the name-to-pid conversion to processes
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: with the same uid as the present process
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -s, --signal <sig> send specified signal
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -q, --queue <sig> use sigqueue(2) rather than kill(2)
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -p, --pid print pids without signaling them
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -l, --list [=<signal>] list signal names, or convert one to a name
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -L, --table list signal names and numbers
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -h, --help display this help and exit
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: -V, --version output version information and exit
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com kill[5462]: For more details see kill(1).
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com systemd[1]: keydb.service: control process exited, code=exited status=1
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com systemd[1]: Unit keydb.service entered failed state.
    Apr 15 18:23:33 rabbit01.nyj.twelvefold.com systemd[1]: keydb.service failed.
    John Sully
    @JohnSully
    Anything interesting in the logs before the first one you sent? Odd its showing the help for the kill command, perhaps its an external signal sent to KeyDB
    smartattack
    @smartattack
    i suspect that is coming from the systemd unit file which has "ExecStop=/bin/kill -s TERM $MAINPID"
    as to what could be triggering it, I am at a loss. I see the same output as from journalctl in syslog, and in the keydb logfile the activity looks normal and then there's a restart logged.
    28951:30636:M 15 Apr 2020 18:23:14.144 - 121 clients connected (0 replicas), 25652796280 bytes in use
    28951:30636:M 15 Apr 2020 18:23:19.209 - DB 0: 119736694 keys (119736692 volatile) in 134217728 slots HT.
    28951:30636:M 15 Apr 2020 18:23:19.209 - 112 clients connected (0 replicas), 25651635808 bytes in use
    28951:30636:M 15 Apr 2020 18:23:24.273 - DB 0: 119732468 keys (119732466 volatile) in 134217728 slots HT.
    28951:30636:M 15 Apr 2020 18:23:24.273 - 77 clients connected (0 replicas), 25650129496 bytes in use
    5466:5466:C 15 Apr 2020 18:23:33.891 # oO0OoO0OoO0Oo KeyDB is starting oO0OoO0OoO0Oo
    5466:5466:C 15 Apr 2020 18:23:33.891 # KeyDB version=5.3.3, bits=64, commit=07cb1f45, modified=0, pid=5466, just started
    John Sully
    @JohnSully
    SIGTERM is signal 15 though, you are receiving 6
    smartattack
    @smartattack
    good point. It looks like if glibc has a memory allocation error the message may by default go to /dev/tty, and not stderr/stdout.
    I am going to try setting an Env variable: "LIBC_FATALSTDERR=1" to see if that helps surface anything.
    will report back here if I am able to find anything useful
    smartattack
    @smartattack
    For reference, that came from here after searching for what can cause a sigabort to be raised: https://stackoverflow.com/questions/32056387/catching-libc-error-messages-redirecting-from-dev-tty
    John Sully
    @JohnSully
    Yes that should be super helpful. I'm debating running the sigsev crash handler on sigterm as well, but it may need some modifications for that purpose
    smartattack
    @smartattack
    I'm a little afraid that glibc may not read that variable due to some code I don't understand having to do with __libc_enable_secure. This is a bit over my head, but anyway I will try this first. If it does not illuminate anything, I can relaunch keydb from a tmux and see if it prints anything there.
    Patryk Kuźmicz
    @jamzed
    I am observing the same issue with SIG ABRT on Ubuntu 18.04, I am not able to reproduce it, it happens from time to time. No more details in logs, only one line in apport.log
    ERROR: apport (pid 27623) Sat Apr 18 17:10:24 2020: host pid 16859 crashed in a separate mount namespace, ignoring
    This is KeyDB 5.3.1 with some patches from 5.3.2.
    Patryk Kuźmicz
    @jamzed
    systemd only reported this:
    Apr 18 17:10:25 xxxxx systemd[1]: keydb-server.service: Main process exited, code=killed, status=6/ABRT
    Apr 18 17:10:25 xxxxx systemd[1]: keydb-server.service: Failed with result 'signal'.
    Apr 18 17:10:45 xxxxx systemd[1]: keydb-server.service: Service hold-off time over, scheduling restart.
    Apr 18 17:10:45 xxxxx systemd[1]: keydb-server.service: Scheduled restart job, restart counter is at 1.
    smartattack
    @smartattack
    I ran it from a tmux session and found this:
    -bash-4.2$ /usr/bin/keydb-server /etc/keydb/keydb-tty.conf keydb-server: fastlock.cpp:436: void fastlock_free(fastlock*): Assertion `(lock->m_ticket.m_active == lock->m_ticket.m_avail) || (lock->m_pidOwner == gettid() && (lock->m_ticket.m_active == lock->m_ticket.m_avail-1))' failed.
    smartattack
    @smartattack
    Please let me know if that helps or if I can provide more information
    smartattack
    @smartattack
    Added as issue #170 in github
    Enea
    @eni9889
    I'm having the same issue
    Any updates on this?
    @smartattack
    @jamzed
    smartattack
    @smartattack
    not yet, no
    Guillaume Lakano
    @lakano
    Hello there ! I'm interested about KeyDB, it's seems amazing! :)
    I would like to known if the "multiple master" feature works with servers in different continent. Imagine 3 servers, America, Europe, Asia, and all the 3 servers are all master and synced. Is it possible ? Is the network latency to sync between each will slow down KeyDB performances in local usage (eg: compared to a single isolated and not synced KeyDB server in each continent )
    Thanks for your help ! :-)
    smartattack
    @smartattack
    @eni9889 @jamzed Looks like there was an update on the case this weekend and the issue is fixed. I'm building the unstable branch and will cut over, today.