These are chat archives for atomix/atomix

10th
Apr 2018
Jordan Halterman
@kuujo
Apr 10 2018 04:44
I think I should be able finish up the last API changes tomorrow: configuring and accessing arbitrary primitives via the REST API. After that, the code needs to settle down for a while. More tests need to be added and the existing tests need to be fixed.
I think I also have some more changes to port over from ONOS: some Netty connection pool changes, efficient iterators for large primitives, streaming APIs for primitives, support for large sets of Raft events.
Johno Crawford
@johnou
Apr 10 2018 07:21
@kuujo so let's say I create a data only cluster, what type of behaviour might be expected in comparison to a cluster with core nodes? Things like network partitions are not handled / brain split? What else?
Johno Crawford
@johnou
Apr 10 2018 07:36
No coordination primitives I imagine eg. distributed locks
Johno Crawford
@johnou
Apr 10 2018 08:48
would also be nice to have a clear explanation of how the above comes into play with say MultiPrimaryProtocol vs RaftProtocol
Jordan Halterman
@kuujo
Apr 10 2018 13:47
Yep totally. Some day soon we’ll have documentation on par with the old documentation.
Jordan Halterman
@kuujo
Apr 10 2018 14:10

Basically, a DATA node only cluster uses an eventually consistent group membership protocol that results in an eventually consistent primary election protocol which allows split brain for all primitives.

A cluster with CORE nodes will always have at least one Raft partition that can tolerate partitions and is used for primary election. It’s still possible in primary-backup partition for two nodes to believe themselves to be the primary at the same time and accept writes, so coordination primitives should use a Raft partition group if CORE nodes exist.

But primary-backup partitions and multi-primary protocols allow for much more scalable, in-memory replication, with a configurable number of backups, synchronous or asynchronous replication, able to scale down to a single DATA node, able to rebalance more efficiently, etc.

Johno Crawford
@johnou
Apr 10 2018 14:45
so if I wanted a distributed map between multiple services i'd maybe have three (allowing for better consensus) core nodes, three data nodes and define the map something like this?
ixConcurrentMap(Atomix atomix, String id) {
        this.cache = atomix.<K, V>consistentMapBuilder(id)
                .withSerializer(createSerializer())
                .withProtocol(MultiPrimaryProtocol.builder()
                        .withMaxRetries(5)
                        .withConsistency(Consistency.LINEARIZABLE)
                        .withReplication(Replication.SYNCHRONOUS)
                        .withRecovery(Recovery.RECOVER)
                        .build())
                .build();
    }
Johno Crawford
@johnou
Apr 10 2018 15:30
Backups essentially meaning copies of partitions which can be used if a node goes down with specific data?
Johno Crawford
@johnou
Apr 10 2018 18:46
@kuujo why did you put rest in core? :confused:
Jordan Halterman
@kuujo
Apr 10 2018 18:47
only annotations
jaxrs annotations that is
Johno Crawford
@johnou
Apr 10 2018 18:48
ah I jumped to conclusions, I saw the package name and was like wtf
Package javax.ws.rs - Oracle Help Center
Package javax.ws.rs High-level interfaces and annotations used to create RESTful service resources.
sorry :)
Jordan Halterman
@kuujo
Apr 10 2018 19:07
hmm… I may want to move them back to the REST package though I dunno
would just need to add another service loader for REST resources
Johno Crawford
@johnou
Apr 10 2018 19:12
ah interfaces as well
otherwise I would have said can you mark it provided scope
Jordan Halterman
@kuujo
Apr 10 2018 19:13
hmm… that may work?
Johno Crawford
@johnou
Apr 10 2018 19:26
Depends how you plan to use it but yes that's one nice thing about annotations
Jordan Halterman
@kuujo
Apr 10 2018 19:36
this is just about everything that’s used from that dependency and a good example of how it’s used:
@POST
@Path("/inc")
@Produces(MediaType.APPLICATION_JSON)
public void incrementAndGet(@Suspended AsyncResponse response) {
every annotation and interface is from that dependency
but this class is only loaded by atomix-rest
meh
I can’t decide whether the resource classes should be in core or not
on one hand, they’re specific to individual primitive implementations that are defined in core, and it’s nice to allow the PrimitiveType to define a REST resource for primitives as well
on the other hand, it uses REST specific annotations, but they’re just annotations and interfaces, no actual functional dependencies are added unless the atomix-rest module is used
Johno Crawford
@johnou
Apr 10 2018 19:47
Are those methods used when the rest module isn't in use
Jordan Halterman
@kuujo
Apr 10 2018 19:56
nope
Johno Crawford
@johnou
Apr 10 2018 20:41
Provided or even optional dependency would work then imo