These are chat archives for atomix/atomix

26th
Jul 2017
Andrew Audibert
@aaudiber
Jul 26 2017 00:13
We can't apply 101, 102, 103 unless we first apply a combination of snapshot+entries that represents entries 1-100. So after applying 101, 102, and 103, the state machine must be in a state representing entries 1-103. After we install snapshot 103, we are still in a state representing entries 1-103. If installing a snapshot didn't change the state machine's state, why do we need to install it?
Jordan Halterman
@kuujo
Jul 26 2017 00:16
You're not applying entries 1-103. You're applying entries e.g. 101-103. Assuming a segment is 100 entries, after the logs are compacted, there are no entries at indexes 1-100 any more. The log effectively starts at 101, but the snapshot represents entries 1-103, so the snapshot must override entries that have been applied prior to its
The log example I described above is an entire log. The problem is that those earlier entries no longer exist. This is the reason we take snapshots.
Normally the snapshot is effectively the first entry in the log. The server will have a snapshot for entries up to index 100 and the log will start at 101. But to reduce the overhead of snapshotting, Atomix allows snapshots to be taken at different points in the log. They just have to overwrite entries previously applied to the state machine to act as if they're the beginning of the log for that state machine.
Andrew Audibert
@aaudiber
Jul 26 2017 00:28
Ah ok, that makes sense now, thanks for explaining. To clarify, would snapshot installation only happen when the server starts or falls behind?
Jordan Halterman
@kuujo
Jul 26 2017 00:36
Yep exactly. Always when a server is restarted. Or if a follower falls too far behind the leader for the leader to be able to send entries (the leader already compacted the entries from its logs) it will send the snapshot over the network.
Otherwise, during normal operation the state machine will always be ahead of snapshots