If you somehow suffer a split-LAN event, how do you determine which is the official and which the split, and how can you remerge them?
The way we solve this is to configure every node to know every other node -- this way everyone knows how big the cluster is, and knows if it's able to talk with more than half of it. If a node can't talk with more than half of the cluster, then it will refuse to "stand up" to be the leader.
Would bedrock be suitable for synchronizing a large number of instances? say 100-200, where the DB is quite simple however (low traffic and amount of data)
I haven't tried that many, but I can't think of any reason why this wouldn't work. The one complication is, as mentioned above, the design requires that every node know every other node ahead of time -- new nodes currently can't be added or removed without restarting the cluster.
Is Bedrock different than RQLite in some important feature? The plugins like cache and job queue are valuable, for instance.
Honestly I don't know much about RQLite, sorry!
I am trying to compile Bedrock on Mac OSX, and I get the following error: libstuff/libstuff.h:44:59: fatal error: pcrecpp.h: No such file or directory
include <pcrecpp.h> // sudo apt-get install libpcre++-dev
^compilation terminated.
make: * No rule to make target .build/main.d', needed by.build/main.o'. Stop.
brew is always breaking, and I feel like I need to reinstall my entire OS whenever anything breaks.
"pcrecpp::RE::Init(std::cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, pcrecpp::RE_Options const)", referenced from:
SParseHTTP(char const, unsigned long, std::cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SString, STableComp, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, SString> > >&, std::cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) in libstuff.a(libstuff.o)
BedrockPlugin_Jobs::_isValidSQLiteDateModifier(std::cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) in libbedrock.a(Jobs.o)
BedrockPlugin_MySQL::onPortRecv(STCPManager::Socket*, SData&) in libbedrock.a(MySQL.o)
ld: symbol(s) not found for architecture x86_64
DB plugin to make leaders just refuse writes escalated from permanent followers.
Permafollower, such as shown here: https://github.com/Expensify/Bedrock/blob/cf2c36027f569df277d3f97ea08ff8a917ad014a/sqlitecluster/SQLiteNode.cpp#L370
DB.cpp to a new file SecureDB.cpp in /plugins -- that'll ensure it just compiles easy. Later when you have proven it out, you'll likely want to move your SecureDB.cpp plugin into your own private repo, but I'd suggest skipping that to start.
SecureDB plugin, launch Bedrock with the -plugins SecureDB parameter to enable it (and notably, don't enable DB, as that doesn't have the security check you want)
Permafollower, but it is as simple as making the -priority of the follower 0 in the follower's command line, and then adding ?Permafollower=true to the node definition in -peerList on all of the full nodes. So, like -peerList node1:9001,node2:9002?Permafollower=true would indicate that node2:9002 is a Permafollower. This is necessary to configure on the leader as well as the follower, because obviously the leader can't trust the follower to advertise its own permafollower status (given that you are using this to identify "untrusted" nodes).
@quinthar Pardon me for asking a trivial question. I tried creating the SecureDB.cpp as suggested that is needed in this logic (I hope I located it in the right location)
if (SStartsWith(upperQuery, "INSERT ")) {
if(peer->params["Permafollower"] == "true") {
STHROW("400 Unauthorized");
} else
response["lastInsertRowID"] = SToStr(db.getLastInsertRowID());
}
... but sadly, am having trouble getting the code to understand "peer" variable
I tried this
Peer* peer = getPeerByID(command.initiatingPeerID);
(and I included some .h files at the top of the code)
This is in the BedrockPlugin_DB::processCommand(...) function but does not compile (that is how I came to know about "peer" variable) .
My C++ is quite rusty and I have not fully gone thru the code -- just kind of intuitively understanding what is to be done.
Hi all, I currently run a Discord bot with a SQLite backend database. It was working fine until I enabled sharding, meaning there are multiple threads writing/reading from the same SQLite database. I've enabled WAL mode however I still get SQLITE_BUSY error codes.
I stumbled upon Bedrock, so I figured I could give it a shot. I was able to build and run Bedrock successfully, pointing to my existing SQLite file, however, because there's no Java SDK for Bedrock I decided to give the MySQL plugin a shot. This is where the horror story starts, I'm only able to make SQL queries by connecting to Bedrock via the MySQL CLI client, but not if I'm connected via my bot using the ORM.
Am I out of luck here?