so here is the current list of feedback on the PR
- report errors for invalid CLI invocations (e.g. like leaving off log name --> segfault)
- have
dumpreport trim/filled position states - clean up the commit log a bit after all the dev work is done. for example, it's confusing to see all the intermediate development steps (wip, then remove rename, etc...) or add testing then fix testing. i'd squash the commits down into logical commits. unfortuantlely there isn't any right way to do this, it's just a look and feel kind of thing.
other than those things, the PR looks really good
like to test it?
also when I fill/trim past the end and dump I get ENOENTs starting at the end of the written log
vsui zlog (cli) $ ./bin/zlog --backend lmdb --db-path ../lmdb/ log create newlog
zlog.head.2aa01f68-8fdf-4c38-bbc0-0c9c33a617d3
zlog.data.2aa01f68-8fdf-4c38-bbc0-0c9c33a617d3
vsui zlog (cli) $ ./bin/zlog --backend lmdb --db-path ../lmdb/ log append newlog a
vsui zlog (cli) $ ./bin/zlog --backend lmdb --db-path ../lmdb/ log append newlog b
vsui zlog (cli) $ ./bin/zlog --backend lmdb --db-path ../lmdb/ log append newlog c
vsui zlog (cli) $ ./bin/zlog --backend lmdb --db-path ../lmdb/ log trim newlog 1
vsui zlog (cli) $ ./bin/zlog --backend lmdb --db-path ../lmdb/ log trim newlog 5
vsui zlog (cli) $ ./bin/zlog --backend lmdb --db-path ../lmdb/ log dump newlog
0: 61
1: invalidated
2: 63
log::Read -2
yeah to test it
what happens if you try to read 10000
or whatever
like read 20
should I add a read command to just print a specific entry
should I add a read command to just print a specific entry yeh that'd be awesome
you could even report "pos: object not found (see cruzdb/zlog#316)" haha
i think it is really cool that you reached out in twitter DM about OSS and then go involved and put together a non-trivial patch. definitely feel free to say no, but if you don't mind, I'd like to tweet out a little screenshot of DM and the subsequent PR to show #opensource how easy it is to get involved with some projects.
What do you think?
I think that implanting delete / gc would be helpful and would also require more design work to make sure it’s correct
Do you have any experience with TLA+?
Lemme look at my notes. I think there are some other things that I’m not thinking of right now
Would be down try to learn TLA+ if there’s anything that needs to be done with it, it sounded really cool
yeh that's about my level of knowledge too. i've hacked with it a bit, but i'm not very good. one goal i've had is to formally model zlog to show that it is correct. this is useful to build confidence in the implementation, but also to test out new optimizations in the model before committing to building them in C++. so that's definitely a project that is cool (plus formal modeling a hot topic right now).
so i think (1) tla+ modeling (2) or something like: log deletion, garbage collection, monitoring tools, integrations... generally working on something to advance zlog to its 1.0 goal or (3) cruzdb type stuff. i can expand on (2) tomorrow--i just need to go look at my notes
i will. i just bought this book a few days ago: https://www.amazon.com/Practical-TLA-Planning-Driven-Development/dp/1484238281
i'm working through it
you can also consider log delete / gc to be absolutely necessary things that need to be done, but they will require some design work--not as straightforward as building out a CLI.
but that might be a benefit :)
the key thing with delete is that i think we want to separate it into two actions. (1) delete: which is a logical operation and (2) garbage collection where we actually remove the log storage. For (1) i think we want to (a) mark in the metadata of the head object that the object is deleted and then (b) delete the link object. If we add a creation timestamp or "last opened" timestamp to the head object, then for (2) we can add a mode to the CLI to look for deleted objects older than N days and actually perform the deletion.
