These are chat archives for atomix/atomix

31st
Jan 2017
Jordan Halterman
@kuujo
Jan 31 2017 11:28

The refactoring for Copycat 2.0 is just about complete. But there's still a long way to go in terms of stability, and it will likely take some time considering my limited availability these days.

None of the critical components of the Raft implementation were touched, but plenty was still changed:

  • The Catalyst dependency was removed and buffers were moved into Copycat
  • The Catalyst Transport API was replaced with a Copycat specific Protocol API that facilitates plugging in different types of protocols, e.g. TCP, HTTP, or web sockets (for Atomix)
  • Serialization has been significantly simplified. Indeed, Copycat no longer even has a serialization abstraction. State machine operations are simply submitted as Buffers and state machine input, output and events are Buffers as well. This allows for better memory management and for the user (and Atomix) to handle serialization themselves. This also significantly reduces the amount of unnecessary serialization/deserialization that takes place inside the server.
  • The Copycat log was completely rewritten to support a single writer and multiple readers using locks. This has relaxed the threading mode inside the server and allows leaders to replicate to followers in parallel. It also removes the need for indexes in the log since each follower has a unique LogReader that tracks that node's position in the leader's log. Log compaction was also redesigned to reduce memory consumption and improve efficiency.
  • State machine Commands and Querys were replaced with Buffers. Copycat is unopinionated about how state machines handle those operations.

So, overall Copycat has been simplified significantly. And while Copycat has become decidedly less opinionated, Atomix will become a bit more opinionated in some respects. That phase of refactoring is currently in progress and I should start pushing to the Atomix 2.0 branch some time this week. As I mentioned previously, Atomix 2.0 will have:

  • More efficient management of memory in resources
  • Pluggable protocols, including support for HTTP and web sockets and interaction between HTTP, web sockets, and the native TCP protocol
  • Faster serialization
  • Significantly better configuration (likely YAML) for standalone servers
  • Command line utilities for interacting with resources
  • Easier extensibility
Jon Hall
@jhall11
Jan 31 2017 23:56
Any thoughts on atomix/copycat#260 It looks like the ONOS project will need a bugfix for this bug