These are chat archives for atomix/atomix

28th
Jan 2016
Richard Pijnenburg
@electrical
Jan 28 2016 08:55
see you guys have been busy with the documentation. its looking very good.
Jordan Halterman
@kuujo
Jan 28 2016 08:56
yep
just gotta keep hackin on it
shit takes forever, but I write too much
Richard Pijnenburg
@electrical
Jan 28 2016 08:56
hehe yeah indeed
Jordan Halterman
@kuujo
Jan 28 2016 08:56
just trying to get the algorithm documentation done so I can publish it to the Raft community for review before either project is released
Richard Pijnenburg
@electrical
Jan 28 2016 09:01
ahh nice :-)
yeah its quite hard to document it correctly i think
Jordan Halterman
@kuujo
Jan 28 2016 09:06
totally
the hard part I think is imagining the questions someone with no knowledge of the system might ask
Richard Pijnenburg
@electrical
Jan 28 2016 09:07
defo
Jordan Halterman
@kuujo
Jan 28 2016 09:11
The documentation on the log and log compaction takes up almost half of this big ass page: http://atomix.io/copycat/user-manual/internals/#the-copycat-log
But that’s the portion of the documentation that’s most critical for review by fellow Raft peeps since it’s especially unique compared to anything suggested anywhere in the Raft literature. Even Diego’s dissertation only suggests snapshots, log cleaning, and LSM trees for logging/compaction in Raft, but Copycat implements a custom algorithm that uses the good parts from all three. Logs are incrementally compacted, but snapshots are supported on top of that for simpler state machines. There are lots of crazy complexities in managing that process though, particularly when you consider things like adding nodes to the cluster.
Jordan Halterman
@kuujo
Jan 28 2016 09:29
I’d be interested to know how you fare if you do attempt to tackle that documentation. This is where it’s difficult for me to imagine the question a user might have. The documentation on the Copycat internals is certainly geared towards a knowledge of the Raft consensus algorithm, but you never know who could be reading and it should still be accessible at least at a high-level for non-Raft-experts.
Richard Pijnenburg
@electrical
Jan 28 2016 09:36
I'll defo read it and provide feedback on it
Jordan Halterman
@kuujo
Jan 28 2016 10:44
Well, I'm reading it and it's still definitely not very good. Probably will take a couple more days to get it where it should be
Off to take a nap!
Richard Pijnenburg
@electrical
Jan 28 2016 19:06
' 8.2.9 Handling membership changes' second part ends a bit weird
Jordan Halterman
@kuujo
Jan 28 2016 19:07
oh yeah I noticed that last night too
clearly I must have been typing something and then fell asleep :-P lol jk
Richard Pijnenburg
@electrical
Jan 28 2016 19:07
hahaha :p
or got distracted
Richard Pijnenburg
@electrical
Jan 28 2016 19:35
time to head home.
had enough for today
lol
Jordan Halterman
@kuujo
Jan 28 2016 19:45
Haha