Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    David Barrett
    @quinthar
    But you can fix that by just executing the schema change using the sqlite3 binary
    It's not great, and is totally fixable, but our databases are far too large to do live schema changes in production so it hasn't been an issue for us (especially given the easy workaround)
    Ryan Bastic
    @rbastic
    Thank you. One more bug report: if you have the database still open with sqlite3 bedrock.db, then run ./bedrock, nc localhost 8888 will not work. If you close sqlite3 bedrock.db, then run ./bedrock - nc localhost 8888 works. So it seems the listener fails silently when the database is open elsewhere.
    Ryan Bastic
    @rbastic
    I was just able to create a small table with a json column and use some json functions. Very cool! I'll continue to R&D further now.
    Ryan Bastic
    @rbastic
    Typo report: 'more detailed comparision against MySQL' on bedrockdb.com - should be 'more detailed comparison'.
    David Barrett
    @quinthar
    Ah, thanks!
    Ryan Bastic
    @rbastic
    Yw! I'm sure I'll have more questions soon. Probably won't get to play with Bedrock more until tonight/tomorrow.
    Caleb Johnson
    @calebj

    I'm having a hard time finding the sqlite branch referenced here (source):

    But lesser known is that there is a branch of SQLite that has page locking, which enables for fantastic concurrent write performance.

    Which one is it, and why is it so underrated? I don't see any obvious candidates on https://www.sqlite.org/src/brlist

    danielk1977
    @danielk1977
    Caleb Johnson
    @calebj
    Excellent, thank you
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    hello, I just found Bedrock and am very interested in it. Perhaps this was discussed before, sorry for asking something perhaps obvious, but does Bedrock run fine in Raspberry Pi's? Does it need any particular version of the OS?
    I want to have a network of devices that come and go at will (get turned off), in a local LAN. I want them to resynch so that any one of them can become the new Leader. I want to be able to make some nodes more likely to be leaders (since they will be full PC's), but I believe that is already a feature in Bedrock. My concern is what happens when only one device is left on, will it stop working due to lack of concensus? or can this be set so even if a single node is left it can continue working as the Leader?
    David Barrett
    @quinthar
    hello @mruizcamauer_twitter! I think it should work just fine on a Pi; it doesn't do anything fancy.
    As for synchronization, here's an overview of how it works: https://bedrockdb.com/synchronization.html
    That said, any kind of synchronization requires a quorum of nodes to be online in order to commit anything -- otherwise you open yourself up to a "split brain" scenario (where something causes the cluster to break into two parts, each of which commits transactions that conflict with the other, "forking" the cluster).
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    What if you have a system where you may have one-N devices on at any time? The second one to come on should assume the first one is the leader and get an update from it. Can it be configured to work with only one device on?
    Is Bedrock different than RQLite in some important feature? The plugins like cache and job queue are valuable, for instance.
    Would bedrock be suitable for synchronizing a large number of instances? say 100-200, where the DB is quite simple however (low traffic and amount of data)
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    If you somehow suffer a split-LAN event, how do you determine which is the official and which the split, and how can you remerge them?
    @quinthar how do you resolve this situation? how do you find the two halves?
    David Barrett
    @quinthar

    If you somehow suffer a split-LAN event, how do you determine which is the official and which the split, and how can you remerge them?

    The way we solve this is to configure every node to know every other node -- this way everyone knows how big the cluster is, and knows if it's able to talk with more than half of it. If a node can't talk with more than half of the cluster, then it will refuse to "stand up" to be the leader.

    Would bedrock be suitable for synchronizing a large number of instances? say 100-200, where the DB is quite simple however (low traffic and amount of data)

    I haven't tried that many, but I can't think of any reason why this wouldn't work. The one complication is, as mentioned above, the design requires that every node know every other node ahead of time -- new nodes currently can't be added or removed without restarting the cluster.

    Is Bedrock different than RQLite in some important feature? The plugins like cache and job queue are valuable, for instance.

    Honestly I don't know much about RQLite, sorry!

    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    thanks for the replies!
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter

    I am trying to compile Bedrock on Mac OSX, and I get the following error: libstuff/libstuff.h:44:59: fatal error: pcrecpp.h: No such file or directory

    include <pcrecpp.h> // sudo apt-get install libpcre++-dev

                                                           ^

    compilation terminated.
    make: * No rule to make target .build/main.d', needed by.build/main.o'. Stop.

    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    I installed the PCRE libraries, but still no go...
    David Barrett
    @quinthar
    Ah yes, getting anything to compile in OSX is always a pain. Every time the OS updates, it seems to break all the paths
    I have had so much trouble trying to basically do any development natively in OSX that I've just kind of given up. brew is always breaking, and I feel like I need to reinstall my entire OS whenever anything breaks.
    The upshot is: yes, it compiles under OSX... if you can figure out how.
    But it'll probably stop compiling when you upgrade the OS again.
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    any binaries already compiled? I need to take it for a spin to decide if we can use it in a really neat application...
    David Barrett
    @quinthar
    Well where would you be deploying it? Probably not on OSX, right?
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    no, RPI
    raspberry pi
    if we can compile it for that, of course...
    David Barrett
    @quinthar
    yes, any Linux environment it should compile much cleaner
    I'd suggest just testing it on the environment that you intend to use it
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    One important thing is determining how quickly you can add new nodes, given a tiny db (3mb)... I imagine it would be a very quick synchronization, but if you had 50 clients, would the 51st be fast? would turning on all 50 be fast?
    David Barrett
    @quinthar
    Well to clarify, the design isn't that you can just add new nodes automatically -- currently it requires that you reboot all the existing nodes to add one more. This is because every node needs to know how big the cluster is in order to know if it has quorum or not.
    Additionally, it's not designed to "bootstrap" by downloading the database from another node; it instead downloads the most recent backup of the database (eg, from S3) and then synchronizes the most recent commits from a peer. (This is because each node only keeps the most recent N commits, not all commits back to the start of time.)
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    but copying the db from the leader would not be an option to getting the latest db?
    "rebooting", how long does that take?
    David Barrett
    @quinthar
    Well it depends on database size; it's as fast as it takes to download your database from S3.
    Marcelo Ruiz Camauër
    @mruizcamauer_twitter
    !ok, it's all LAN, so it should be very fast1
    managed to advance in compilation a little, it was missing paths to PCRE libraries (at least mine)... stuck now in another place
    Undefined symbols for architecture x86_64:
    "pcrecpp::RE::Init(std::cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, pcrecpp::RE_Options const)", referenced from:
    SParseHTTP(char const
    , unsigned long, std::
    cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SString, STableComp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, SString> > >&, std::cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) in libstuff.a(libstuff.o)
    BedrockPlugin_Jobs::_isValidSQLiteDateModifier(std::
    cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in libbedrock.a(Jobs.o)
    BedrockPlugin_MySQL::onPortRecv(STCPManager::Socket*, SData&) in libbedrock.a(MySQL.o)
    ld: symbol(s) not found for architecture x86_64
    TAD Master
    @teamtad_twitter
    Hi. I am new to Bedrock -- I find the points for it quite justified, so am thinking of using it for my project. However, I have the following situation -- and I hope some guru here would let me know if that is possible. I want to give out an sqlite database to anyone who wants to read it from their part of the world -- So I want a master node with a set of readonly slave nodes.. That is the kind of distribution of the data I want. Basically, writing into the database would be only from the master (or some set of known nodes); but only reading should be possible on other nodes. Is this kind of distribution architecture possible with Bedrock? Thanks in advance.
    David Barrett
    @quinthar
    @teamtad_twitter Yes, that's exactly how the design currently works! At any point in time there is one leader, and any number of followers. The leader is elected by a quorum of "full nodes" -- ie, anybody who you designate as a viable leader. But you can also designate any number of permanent followers, and thus they will not participate in the leader election (and will never become leaders). Rather, they will remain permanently read-only followers that escalate any attempt to write to the leader.
    If the goal is to further constrain followers to not even escalate write commands to the leader, that isn't something currently supported out of the box, but would be pretty easy to do by creating a custom version of the DB plugin to make leaders just refuse writes escalated from permanent followers.