These are chat archives for atomix/atomix

14th
Aug 2017
Jordan Halterman
@kuujo
Aug 14 2017 18:14

@garyttierney Good questions. releaseing an entry doesn't mean it will be immediately removed from disk. That would be an expensive operation, and it would always be plagued with race conditions. There's always the possibility a state machine could release an entry and crash before it's actually removed from disk.

So, instead, state machines must be deterministic, including in the way they release commits. Yes, entries that haven't yet been removed from disk will be replayed at startup. Eventually, when the log rolls over to a new segment it will rewrite the old segment and remove released entries. So, the entries that are replayed at startup depends on whether the logs have been compacted, but assuming the state machine only ever released entries that are no longer needed that shouldn't matter (presumably they're overwritten by some later changes).

Atomix 2 uses only snapshotting for log compaction. It splits the log into logical partitions, with a partition per state machine and a snapshot per partition so it can still compact logs efficiently. But state machines are significantly simpler in that they'll just periodically write their state to disk, and the snapshot will be reinstalled at startup. No need to manage the entries on disk and ensure tombstones are retained.

and as for Atomix 2, I’ll be starting this week on the remaining refactoring of high level distributed systems primitives, partitioning, cluster management, etc
Gary Tierney
@garyttierney
Aug 14 2017 19:47
@kuujo great answer, thank you!