These are chat archives for atomix/atomix

24th
Mar 2017
Stephen Carman
@hntd187
Mar 24 2017 11:30
I don't get it at all
I can't get a simple custom resrouce to work at all
I have an AbstractResource, a ResourceStateMachine, a ResourceFactory, but I can't get a working example
whenver I try and submit anything to the cluster i get an application_error
and no stack trace otherwise
I am literally trying to just add numbers to List
and return said List
I dunno, maybe it's Scala?
Himanshu
@himanshug
Mar 24 2017 14:51
@hntd187 are you trying to write a custom resource or just want to see an example. if you just want working examples then take a look at https://github.com/atomix/atomix/tree/master/examples . I have personally run DistributedGroup example and it worked fine (I just hard coded my server addresses in the code instead of taking them from args) . OTOH if you want to write your own resource, then take a look at GroupState, MembershipGroup which are main classes for DistributedGroup implementation.
Stephen Carman
@hntd187
Mar 24 2017 14:56
Yea, I'm trying to write a custom resource
So I can create the resource
but any operations on it
do not work
I can run examples fine, but I am trying to create things beyond the examples
Himanshu
@himanshug
Mar 24 2017 15:24
hmmm... i've only written own state machine on copycat ... so don't have custom resource example but one thing, that I see, is easy to miss is to call builder.addResourceType(..) or builder.withResourceTypes(..) to register your things... hth
Stephen Carman
@hntd187
Mar 24 2017 17:17
I have that
so I am registering my types
I'm also kind of confused that if I want an object to persist
the object is meant to actually live in the state machine correct?
Jordan Halterman
@kuujo
Mar 24 2017 17:18
Yeah, I’m working on removing the need to register resources. Will probably just use ServiceLoader instead.
If you want to persist an object, typically you submit it in a command to the state machine, and it’s held in memory in the state machine. This serves a pretty specific use case for in-memory state machines, but doesn’t allow for building larger databases. That requires a “persistent state machine”, which Copycat currently doesn’t support. A persistent state machine would allow the state machine to write to disk. That can be done currently, it’s just really inefficient.
To be clear, when you submit a command within a resource, that command is being persisted by Copycat.
So, you’re typically submitting create/update/delete type commands for an object, and Copycat persists and replicates those commands in a transaction log.
Stephen Carman
@hntd187
Mar 24 2017 17:45
So @kuujo atomix is bad if I wanted to build a very consistent in-memory DB?
Jordan Halterman
@kuujo
Mar 24 2017 17:49

No, that's what it's built for! It's writes changes to disk, but state is held in memory to serve reads more quickly. The disk is needed for strong consistency.

Really, DistributedMap is a strongly consistent, in memory key-value store with a Java Map-like interface.

For scalability, you have to partition the cluster. That feature is not in Atomix right now, but it's been done successfully and will eventually make its way back in.

Himanshu
@himanshug
Mar 24 2017 18:19

@kuujo I'm not sure if you noticed the questions are posted before. Can you provide some insights?
(1) CopycatClient is bootstrapped with a list of Addresses. Is it possible to reset the bootstrap addresses on same copycat client object after it went to suspended/closed state?
(2) When using MEMORY storage level, does MetaStore also use MEMORY buffer and <term, vote> is written only in memory? If yes, then isn't it going to be problematic if a server crashes and restarted very quickly while raft leader election was happening... node would forget that it already voted for the term and might give its vote again to some other candidate in same term. no?
(3) For MEMORY storage level, it seems jvm heap memory is used... in this case I can't see the benefit of jvm managing that memory ... wouldn't it be better to use off-heap memory so that GC is not impacted by copycat's buffers?
(4) Atomix DistributedGroup leader election, it appears if a node becomes group leader and then it's AtomixClient goes to SUSPENDED state (for whatever reason, say this particular node is partitioned off). Then this node will continue to think it is the group leader but rest of the nodes in group get a new leader elected. no?
(5) if a CopycatClient on node-A is in CONNECTED state, then say all of raft cluster hosts went down and whole raft cluster is replaced by another set of hosts, node-A's CopycatClient's session goes to UNSTABLE state and stays there indefinitely unless we restart the process at node A. Is my understanding correct?
If yes, then It makes more sense for node-A copycat client to give up after a while and close the session, then application can take note of CLOSED state of copycat client and discover raft cluster nodes again (say from a VIP that always knows about right set of raft cluster hosts) and "reset/reconnect" the copycat client (as asked in Q.1 before) with new raft cluster nodes.

As I said earlier, I'm looking to replace curator/zookeeper in Druid with copycat. Membership and Leader election are very critical part of Druid cluster, due to the large druid community and scale of operations, We would like to understand things upfront so that life is easy later with less (ideally none) surprises :)

It will be great if we can "meet" on hangout where we can talk in person. Thanks.

Stephen Carman
@hntd187
Mar 24 2017 18:44
So sorry, I'm a bit confused @kuujo atomix is what I want, but I can't partition it across nodes just yet? Or do I have to just engineer it a bit more to partition across nodes? I'd ideally like the monotonic consistency so I can put all the nodes behind a load balancer and let anything coming in handle reads