Hello there, I am just starting with Bedrock Jobs and have some questions, I hope this is the right place?
I want my consumers to wait up to 30s for any Job thats with "hello.", so I send this request:
GetJob
name: hello.*
connection: wait
timeout: 30000But this command immediately returns a "404 No job found". Shouldn't it wait 30s before returning?
- can bedrockdb assign priority value dynamically?
- What needs to happen if nodes need to be added to the cluster at a later point in time?
- Is the data for designed to be ephemeral? Or should you always keep the data dir as fresh as possible and not rely on backups when/if a reboot of a node happens?
Connection: wait is currently not working. I believe we broke it when adding some multi-threaded optimizations, and hasn't been rebuilt yet. Is that right @tylerkaraszewski ?
would hope to be able to completely lose the .db file for a particular node when/if a crash happens on an instances or a VM needs a reboot (and be able to easily restore from s3 using an initContainer before starting bedrockdb).
Ah we do have a method to bootstrap from a S3 backup and then catch up, but we haven't automated it to the level you are likely thinking (ie, it doesn't identify the most recent backup and download it automatically). I could imagine you adding this without too much effort however. (I'm not 100% sure if our backup code is open source now that I think about it, we could look into this if you were seriously interested.)
the last question is about priority and whether or not one can be assigned dynamically on startup? So if i have a startup script that assigns a random integer as the priority are there any drawbacks to that?
Yes, that could likely work fine. In practice we find it makes the system easier to manage if the priorities are static, as then you have a better sense of which nodes are doing which without needing to consult the logs.
Would you guys accept pull requests for these types of enhancements? It would obviously have to touch other places in the codebase so just checking before putting in time here.
Yes, PRs very welcome!!
I like #sqlite and works pretty much good with #strapi and #directus
seems to me still better choice than use "cloud db" but I need more instances into 1file - so I searched and found this article https://sqlite.org/forum/info/339d237d6517783a which refers to you
I what I read is pretty great https://bedrockdb.com very good builded project 😃👏
Q) so it is process service above #sqlite connected through port for sync ?
https://bedrockdb.com/multizone.html
Q) it that communications encrypted somehow ?
Q) how colisions are handeled ?
@Nevernown
Problem with NoSQL, like MongoDB, is about how to make those queries with Javascript. I'm currently trying to do some query in Javascript, but I have not got it working it yet. I do know how to do it with SQL.
For MySQL or PostgreSQL, if you sometime get a lot of data, it may happen that you would need to migrate to SQLite to keep costs manageable:
https://news.ycombinator.com/item?id=27673359
Many small queries are efficient in SQLite:
https://sqlite.org/np1queryprob.html
Using SQLite is 35% faster than using filesystem:
https://www.sqlite.org/fasterthanfs.html
If you need global scale with SQLite, there is BedrockDB:
https://bedrockdb.com
https://twit.tv/shows/floss-weekly/episodes/456
For GDPR encrypted stuff, there is Databunker:
https://databunker.org
If MySQL is not your preferred option, you could also look is MySQL compatible Noria any better:
^C
azureuser@Horseportal:~$ mysql -V
mysql Ver 8.0.29-0ubuntu0.20.04.3 for Linux on x86_64 ((Ubuntu))
azureuser@Horseportal:~$
May 16 13:30:14 Horseportal bedrock: xxxxxx (BedrockServer.cpp:2127) _acceptSockets [main] [dbug] Accepting socket from '127.0.0.1:54778' on port 'localhost:3306'
May 16 13:30:14 Horseportal bedrock: xxxxxx (BedrockServer.cpp:2260) handleSocket [socket2] [info] Socket thread starting
May 16 13:30:16 Horseportal bedrock: xxxxxx (STCPManager.cpp:196) shutdown [socket2] [dbug] Shutting down socket '127.0.0.1:54778'
May 16 13:30:16 Horseportal bedrock: xxxxxx (BedrockServer.cpp:2421) handleSocket [socket2] [info] Socket thread complete (0 remaining)
nc localhost 8888
Query: SELECT 1 AS foo, 2 AS bar;
200 OK
commitCount: 1
nodeName: Horseportal
peekTime: 163
totalTime: 323
unaccountedTime: 133
Content-Length: 16
foo | bar
1 | 2
sqlitecluster/SQLitePool.cpp: In constructor ‘SQLitePool::SQLitePool(size_t, const string&, int, int, int, const string&, int64_t, bool)’:
sqlitecluster/SQLitePool.cpp:13:26: error: no matching function for call to ‘max(size_t&, long unsigned int)’
: _maxDBs(max(maxDBs, 1ul)),