These are chat archives for atomix/atomix

21st
Dec 2015
Richard Pijnenburg
@electrical
Dec 21 2015 10:15
Morning all
Richard Pijnenburg
@electrical
Dec 21 2015 10:35
I see you guys have been busy again :-)
Richard Pijnenburg
@electrical
Dec 21 2015 10:41
Looking forward to setup some tests to see how it works.
Want to combine the Distributed topic together with grouping and locking.
Richard Pijnenburg
@electrical
Dec 21 2015 12:08
I'm also wondering that if i have 3 masters with the same code and i start one to create the Distributed topic resource, what will happen if i start the second and 3rd master? will they realize the queue already exists/
i assume only the current leader can start the queue :-)
Richard Pijnenburg
@electrical
Dec 21 2015 13:34
I think having a persistent queue per node would be handy as well? since the actual event storage won't happen in raft it self right?
or is it ?
Richard Pijnenburg
@electrical
Dec 21 2015 18:59
@kuujo i'll be online later again. like to run some idea's through you to see if its even sensible :p
Jordan Halterman
@kuujo
Dec 21 2015 19:27
@electrical sorry I was out and about but back for the rest of the day. I'll be around. To answer your questions...
If three nodes create the same resource (by referencing the same resource key/name) then each node will point to the same replicated object. If node 1 creates a map and puts a value in it, node 2 should be able to create a map with the same name and read the value from node 1. This is true of all distributed resources...
Jordan Halterman
@kuujo
Dec 21 2015 19:35
The Atomix interface exposes two methods for creating resource instances: get and create. Both methods essentially perform a create-if-not-exists type operation to create the resource in the cluster if it doesn't already exist. The difference between get and create is actually in the number of instances that can be created per node. The term "instance" refers to an instance of an object in the typical sense. An instance of a distributed resource is an object in a single JVM. The get method is essentially a per-node singleton method. When a client or replica calls get, it creates the resource in the cluster if it doesn't already exist, and if an instance of the resource has already been referenced by the client/replica then that singleton instance will be returned, otherwise one will be created and cached. Alternatively, the create method allows a single node to create multiple logical instances of a resource. Each time the create method is called, the resource will be created in the cluster of it doesn't exist, and a new Java object will be returned.
The use case for get is to allow multiple threads to create and reference the same resource without having to coordinate with each other. It essentially says, get or create an instance of the resource, whereas create says create a new instance of the resource (though the resource's state machine may already exist)
Each instance of a resource will have a separate logical session to the cluster. This allows e.g. a single node to create 3 instances of the foo lock and for each instance of the lock to behave independently of the other lock instances on the node. In a multithreaded environment this means one thread can acquire the distributed lock without all threads acquiring it, and each thread can acquire the lock independently of the other threads.
Jordan Halterman
@kuujo
Dec 21 2015 19:41
Similarly, several threads can create multiple instances of a queue and consume queued messages independently of one another.
An instance of a resource (a Java object) is effectively a reference to a replicated state machine in the cluster. When a resource is "referenced" for the first time by a client/replica calling create or get, a state machine for that resource is created on each server. When a client/replica calls getfor the first time or calls create, a new logical session is created to allow the replicated state machine to send messages to that specific object. Logical sessions all share the same real underlying session, and so a million distributed objects on a single node would still only require a single keep-alive for the session.
Jordan Halterman
@kuujo
Dec 21 2015 19:46
Also relevant to the use of multiple resources together (topic, membership group, lock), Atomix will order the operations across resources. That is, Atomix is ultimately a single replicated state machine that creates a "child" state machine for each resource, and that ensures that different resources can interact with each other safely. Calling lock() on lock A and then lock on lock
Jordan Halterman
@kuujo
Dec 21 2015 19:55
All state changes for al resources go through the Raft algorithm, and currently all operations except messages sent on the DistributedMessageBus are treated as state changes. So, all messages published to a DistributedTopicare replicated through the Raft algorithm. The long term goal of Atomix is to make this behavior configurable. Resources should support different replication/consistency levels to allow them to be configured for performance and fault tolerance. All current resources aside from DistributedMessageBus provide strong consistency. Ideally, in the future a DistributedTaskQueue should handle only state changes (adding/removing consumers and handling failures) through the Raft algorithm and support configurable behaviors like dispatch and replication factor. I could probably prototype something like this, but it's really outside the scope of the 1.0 release.
On to testing: I haven't gotten into testing and profiling because I finally discovered the bug that I had been chasing for the last week. At least two critical bugs in the Copycat client necessitated a quick rewrite. I am currently in the process of rewriting the client, and I expect that to resolve any known
...remaining issues. The release candidate should be released this week once that pull request is merged and some of the documentation is updated.