Expensify Bedrock/Lobby

David Barrett

@quinthar

This message was deleted

@/all Anybody want some quick cash? Trying to fix a probably obvious (to the right person) makefile issue to get Bedrock compiling on OSX, eager for help! https://www.upwork.com/jobs/~01e7ad7e026a20a5e1

David Barrett

@quinthar

@gsacre Also, when you send the Status command, try connecting to 8000 -- that's the -serverHost, and thus the one that clients should connect to. (Edit: Oh, actually I see you tried that... hm....)

Can you check /var/log/syslog to see if it's giving any clues as to what's going on?

Ryan Bastic

@rbastic

Hello. What version is currently in production? ./bedrock -clean & segfaults for me on Ubuntu 18.04.

I just built off of master

Ryan Bastic

@rbastic

Er, scratch that. 2018-02-22 tag segfaults on Ubuntu 18.04 with -clean &. Building off of master and following the install instructions on Ubuntu 18.04, then attempting to create a table results in:

Query: CREATE TABLE test (foo int, bar int);

502 Query failed
commitCount: 1
error: cannot modify database schema within CONCURRENT transaction

David Barrett

@quinthar

Ah, can you do touch bedrock.db?

Oh, actually wait

Yes, that's a known problem -- when we added multi-write it broke live schema changes

But you can fix that by just executing the schema change using the sqlite3 binary

It's not great, and is totally fixable, but our databases are far too large to do live schema changes in production so it hasn't been an issue for us (especially given the easy workaround)

Ryan Bastic

@rbastic

Thank you. One more bug report: if you have the database still open with sqlite3 bedrock.db, then run ./bedrock, nc localhost 8888 will not work. If you close sqlite3 bedrock.db, then run ./bedrock - nc localhost 8888 works. So it seems the listener fails silently when the database is open elsewhere.

Ryan Bastic

@rbastic

I was just able to create a small table with a json column and use some json functions. Very cool! I'll continue to R&D further now.

Ryan Bastic

@rbastic

Typo report: 'more detailed comparision against MySQL' on bedrockdb.com - should be 'more detailed comparison'.

David Barrett

@quinthar

Ah, thanks!

Ryan Bastic

@rbastic

Yw! I'm sure I'll have more questions soon. Probably won't get to play with Bedrock more until tonight/tomorrow.

Caleb Johnson

@calebj

I'm having a hard time finding the sqlite branch referenced here (source):

But lesser known is that there is a branch of SQLite that has page locking, which enables for fantastic concurrent write performance.

Which one is it, and why is it so underrated? I don't see any obvious candidates on https://www.sqlite.org/src/brlist

danielk1977

@danielk1977

This branch: https://sqlite.org/src/timeline?r=begin-concurrent

Some docs: https://sqlite.org/src/doc/begin-concurrent/doc/begin_concurrent.md

Caleb Johnson

@calebj

Excellent, thank you

Marcelo Ruiz Camauër

@mruizcamauer_twitter

hello, I just found Bedrock and am very interested in it. Perhaps this was discussed before, sorry for asking something perhaps obvious, but does Bedrock run fine in Raspberry Pi's? Does it need any particular version of the OS?

I want to have a network of devices that come and go at will (get turned off), in a local LAN. I want them to resynch so that any one of them can become the new Leader. I want to be able to make some nodes more likely to be leaders (since they will be full PC's), but I believe that is already a feature in Bedrock. My concern is what happens when only one device is left on, will it stop working due to lack of concensus? or can this be set so even if a single node is left it can continue working as the Leader?

David Barrett

@quinthar

hello @mruizcamauer_twitter! I think it should work just fine on a Pi; it doesn't do anything fancy.

As for synchronization, here's an overview of how it works: https://bedrockdb.com/synchronization.html

That said, any kind of synchronization requires a quorum of nodes to be online in order to commit anything -- otherwise you open yourself up to a "split brain" scenario (where something causes the cluster to break into two parts, each of which commits transactions that conflict with the other, "forking" the cluster).

Marcelo Ruiz Camauër

@mruizcamauer_twitter

What if you have a system where you may have one-N devices on at any time? The second one to come on should assume the first one is the leader and get an update from it. Can it be configured to work with only one device on?

Is Bedrock different than RQLite in some important feature? The plugins like cache and job queue are valuable, for instance.

Would bedrock be suitable for synchronizing a large number of instances? say 100-200, where the DB is quite simple however (low traffic and amount of data)

Marcelo Ruiz Camauër

@mruizcamauer_twitter

If you somehow suffer a split-LAN event, how do you determine which is the official and which the split, and how can you remerge them?

@quinthar how do you resolve this situation? how do you find the two halves?

David Barrett

@quinthar

If you somehow suffer a split-LAN event, how do you determine which is the official and which the split, and how can you remerge them?

The way we solve this is to configure every node to know every other node -- this way everyone knows how big the cluster is, and knows if it's able to talk with more than half of it. If a node can't talk with more than half of the cluster, then it will refuse to "stand up" to be the leader.

Would bedrock be suitable for synchronizing a large number of instances? say 100-200, where the DB is quite simple however (low traffic and amount of data)

I haven't tried that many, but I can't think of any reason why this wouldn't work. The one complication is, as mentioned above, the design requires that every node know every other node ahead of time -- new nodes currently can't be added or removed without restarting the cluster.

Is Bedrock different than RQLite in some important feature? The plugins like cache and job queue are valuable, for instance.

Honestly I don't know much about RQLite, sorry!

Marcelo Ruiz Camauër

@mruizcamauer_twitter

thanks for the replies!

Marcelo Ruiz Camauër

@mruizcamauer_twitter

I am trying to compile Bedrock on Mac OSX, and I get the following error: libstuff/libstuff.h:44:59: fatal error: pcrecpp.h: No such file or directory

include <pcrecpp.h> // sudo apt-get install libpcre++-dev

compilation terminated.
make: * No rule to make target .build/main.d', needed by.build/main.o'. Stop.

Marcelo Ruiz Camauër

@mruizcamauer_twitter

I installed the PCRE libraries, but still no go...

David Barrett

@quinthar

Ah yes, getting anything to compile in OSX is always a pain. Every time the OS updates, it seems to break all the paths

I have had so much trouble trying to basically do any development natively in OSX that I've just kind of given up. brew is always breaking, and I feel like I need to reinstall my entire OS whenever anything breaks.

The upshot is: yes, it compiles under OSX... if you can figure out how.

But it'll probably stop compiling when you upgrade the OS again.

Marcelo Ruiz Camauër

@mruizcamauer_twitter

any binaries already compiled? I need to take it for a spin to decide if we can use it in a really neat application...

David Barrett

@quinthar

Well where would you be deploying it? Probably not on OSX, right?

Marcelo Ruiz Camauër

@mruizcamauer_twitter

no, RPI

raspberry pi

if we can compile it for that, of course...

David Barrett

@quinthar

yes, any Linux environment it should compile much cleaner

I'd suggest just testing it on the environment that you intend to use it

Marcelo Ruiz Camauër

@mruizcamauer_twitter

One important thing is determining how quickly you can add new nodes, given a tiny db (3mb)... I imagine it would be a very quick synchronization, but if you had 50 clients, would the 51st be fast? would turning on all 50 be fast?

Where communities thrive

People

Repo info

Activity

include <pcrecpp.h> // sudo apt-get install libpcre++-dev