These are chat archives for atomix/atomix

17th
Jul 2017
@kerbymart
@kerbymart
Jul 17 2017 12:31
Hello is Atomix project and Copycat the same? I mean I've read that Copycat is now Atomix-raft
@kerbymart
@kerbymart
Jul 17 2017 16:02
@kuujo I tested Copycat and works great, very simple to implement a distributed Key Value store and it stores data into a log file, how does Copycat handles log files as it grows, example in real world data store typically data can grow to several MB to Gigabytes in size.
@kerbymart
@kerbymart
Jul 17 2017 19:08
Hello here's my code for Copycat:
String[] mainParts = {"127.0.0.1", "5000"};
    String[] args2 = {"C:\\tmp", "127.0.0.1:5000"};

    Address address = new Address(mainParts[0], Integer.valueOf(mainParts[1]));

    // Build a list of all member addresses to which to connect.
    List<Address> members = new ArrayList<>();
    for (int i = 1; i < args2.length; i++) {
      String[] parts = args2[i].split(":");
      members.add(new Address(parts[0], Integer.valueOf(parts[1])));
    }

    CopycatServer server = CopycatServer.builder(address)
      .withStateMachine(KeyValueStore::new)
      .withTransport(new NettyTransport())
      .withStorage(Storage.builder()
        .withDirectory(args2[0])
        .withMaxSegmentSize(1024 * 1024 * 32)
        .withMinorCompactionInterval(Duration.ofMinutes(1))
        .withMajorCompactionInterval(Duration.ofMinutes(15))
        .build())
      .build();

    server.serializer().register(Put.class, 1);
    server.serializer().register(Get.class, 2);
    server.serializer().register(Delete.class, 3);

    server.bootstrap(members).join();

    server.cluster().onJoin(member -> {
      System.out.println(member.address() + " joined the cluster");
    });

    server.cluster().onLeave(member -> {
      System.out.println(member.address() + " left the cluster");
    });

    while (server.isRunning()) {
      Thread.sleep(1000);
    }
  }
The problem is when running I get this log and clients cannot connect:
03:06:52.158 [copycat-server-/127.0.0.1:5000-copycat] INFO  i.a.c.server.state.ServerContext - /127.0.0.1:5000 - Transitioning to FOLLOWER
03:06:53.448 [copycat-server-/127.0.0.1:5000-copycat] INFO  i.a.c.server.state.FollowerState - /127.0.0.1:5000 - Polling members [ServerMember[type=ACTIVE, status=AVAILABLE, serverAddress=/127.0.0.1:5001, clientAddress=/127.0.0.1:5001], ServerMember[type=ACTIVE, status=AVAILABLE, serverAddress=/127.0.0.1:5002, clientAddress=/127.0.0.1:5002]]
0
Jordan Halterman
@kuujo
Jul 17 2017 20:41

@kerbymart Atomix Raft is basically Copycat 2.0. Originally, Atomix was really developed as a set of state machines and APIs on top of Copycat, but now it's grown to have more protocols and is going through some major refactoring, so all the projects are being developed in he same repo for now.

To answer your question, Copycat uses an incremental compaction algorithm, which you can read about on the website. Basically, state machines are responsible for telling Copycat when it's safe to remove a command from disk (when it no longer contributes to the state machine's state) by releaseing the commit. You can look at the Atomix 1.x state machines for plenty of examples.

This incremental compaction algorithm is simple for simple state machines like a key-value store, but there are a lot of complexities to managing tombstones (deletes), TTLs, and complex writes that can make non-trivial state machines really complex. The need for state machines to do reference counting risks disk not being properly cleaned up, and command configurations for writes and deletes can be difficult to understand.

For this reason, part of the refactoring of Copycat in Atomix 2.0 was moving all state machines to snapshot based compaction only. In addition to snapshots there are a host of improvements to the implementation that make clients much more stable and increase concurrency in clients and servers.

You can see the Atomix 2.0 Raft implementation in use in ONOS right now:
https://github.com/opennetworkinglab/onos/tree/master/core/store/primitives/src/main/java/org/onosproject/store/primitives/resources/impl

Those primitives will soon be moved back into Atomix 2.0

I'll be documenting the improvements in Atomix Raft this week in preparation for a talk on the subject, and I'll make that documentation available somewhere