These are chat archives for atomix/atomix

3rd
Apr 2018
Michael
@mkulak
Apr 03 2018 12:15
Thanks for the answer. For now I'd rather wait for stable 2.1 than use master directly.
abeep
@abeep
Apr 03 2018 16:08
@kuujo
ConsistentMap will overwrite the other record(which ever wins last), so a pessimistic lock needs to be taken on the key to merge, which would degrade performance.
so inorder to avoid pessimistic locking, i was looking for a merge conflict resolution option only when there are conflict.
I am not sure how optimistic locking can be used here.
Write-behind: I want to write the records to rdbms , mapdb etc, so wanted a in memory map with durable capability(datastore can be any)
if you can give me brief pointers, i can try to implement the same.
Also is there any benchmark done for various datastructures and locks
Jordan Halterman
@kuujo
Apr 03 2018 17:15

@abeep there’s no way to implement conflict resolution because by definition there are no conflicts in the consensus algorithm. There’s a global total ordering of events that are serialized through a single node (for each partition), so there are no conflicts to resolve. There’s no bound within which to resolve conflicts either - e.g. conflicting writes that occur at the same logical time. All operations occur at a distinct point in logical time. The only time that conflicts occur is within the context of multiple operations, e.g. within a transaction. In that case it would be possible to resolve conflicting transactions (which are typically accepted/rejected based on locks/isolation levels). But for normal operations, you’d have to first create conflicts to resolve them, and that seems pointless.

Conflict resolution would make sense in the context of EventuallyConsistentMap where timestamps can match for conflicting writes, but we haven’t copied that primitive over to Atomix yet.

Johno Crawford
@johnou
Apr 03 2018 17:19
indeed, no races either, in the sense that if you have two or more nodes inserting the same key into a map (with put) at the same time, only one will return true.
Jordan Halterman
@kuujo
Apr 03 2018 17:20
That said, aside from consensus it is possible for a primary-backup leader to accept writes after it has been superseded by a later term, and it could make sense to resolve conflicts when we send a snapshot to the new leader. But currently that isn’t being done.
Johno Crawford
@johnou
Apr 03 2018 18:19
@kuujo atomix/atomix#458 is ready for round two
Jordan Halterman
@kuujo
Apr 03 2018 18:40
:+1:
Johno Crawford
@johnou
Apr 03 2018 18:40
i'll rebase
Jordan Halterman
@kuujo
Apr 03 2018 18:43
I’m also almost done with what should be the last big change: adding support for configuration files. It has not been a trivial task. But I think it’s worth it to make the standalone servers more useful.
Johno Crawford
@johnou
Apr 03 2018 18:46
rebased
Johno Crawford
@johnou
Apr 03 2018 19:59
@kuujo i just took latest master and it's blowing up when creating an atomix map
i had
   this.cache = atomix.<K, V>consistentMapBuilder(id)
                .withSerializer(createSerializer())
                .withConsistency(Consistency.LINEARIZABLE)
                .withPersistence(Persistence.PERSISTENT)
                .withReplication(Replication.SYNCHRONOUS)
                .withRecovery(Recovery.RECOVER)
                .withMaxRetries(5)
                .build();
replaced with
        this.cache = atomix.<K, V>consistentMapBuilder(id)
                .withSerializer(createSerializer())
                .withProtocol(RaftProtocol.builder()
                        .withMaxRetries(5)
                        .build())
                //.withConsistency(Consistency.LINEARIZABLE)
               // .withPersistence(Persistence.PERSISTENT)
               // .withReplication(Replication.SYNCHRONOUS)
               // .withRecovery(Recovery.RECOVER)
              //  .withMaxRetries(5)
                .build();
Caused by: java.lang.NullPointerException
    at io.atomix.core.map.impl.ConsistentMapProxyBuilder.buildAsync(ConsistentMapProxyBuilder.java:63)
    at io.atomix.primitive.DistributedPrimitiveBuilder.build(DistributedPrimitiveBuilder.java:196)
    at com.sulake.orbit.cluster.AtomixConcurrentMap.<init>(AtomixConcurrentMap.java:52)
    at com.sulake.orbit.cluster.AtomixClusterPeer.lambda$getAtomixCache$1(AtomixClusterPeer.java:121)
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
    at com.sulake.orbit.cluster.AtomixClusterPeer.getAtomixCache(AtomixClusterPeer.java:121)
guess PartitionGroup<?> partitions = managementService.getPartitionService().getPartitionGroup(protocol); returns null
ah works with MultiPrimaryProtocol
Jordan Halterman
@kuujo
Apr 03 2018 21:13
Hmm yeah this is because the cluster now allows DATA node only clusters. The cluster has to be configured with CORE nodes and RaftPartitionGroups to use RaftProtocol. Need to work on the configuration to make sure it provides sensible defaults. The cluster is getting kind of complicated to configure.
Johno Crawford
@johnou
Apr 03 2018 21:34
aha okay
Johno Crawford
@johnou
Apr 03 2018 21:48
of course, perhaps that could be put in an error message?
Andrew Audibert
@aaudiber
Apr 03 2018 23:46
@kuujo I’m working on upgrading a debugging tool from Copycat to the latest and greatest Atomix-Raft. In Copycat it would connect to my cluster as a passive node and run a basic state machine that would echo out the current state of the log from Copycat/Raft’s perspective. I’m having trouble getting the tool to join as a passive node in Atomix master - it looks like setting the node type is deprecated now: https://github.com/atomix/atomix/blob/master/protocols/raft/src/main/java/io/atomix/protocols/raft/RaftServer.java#L604. Is there another way to join as a passive node, or do you have a suggestion for another way to read the state of the raft logs without fully joining the cluster?