These are chat archives for atomix/atomix

1st
Jul 2016
Gardner Vickers
@gardnervickers
Jul 01 2016 02:04
Ah nice! I’ve actually been working on using Copycat with Clojure to to allow multimethod based distributed state machines. Using multimethods for state machines is a common pattern in Clojure-land, and I have an implementation using a Copycat state machine with one Command and one Query type. The Command type expressing a dispatch “value” (anything serializable that the multimethod can dispatch off), the return of it mutates the Copycat state machine. The Query just returns the whole state of the Copycat state machine.
For the non-clojure folks, The TL;DR for mutlimethods is arbitrary function dispatch, not limited to type-based polymorphism
The Copycat codebase has been a dream to work with. Thanks for your hard work!
Gardner Vickers
@gardnervickers
Jul 01 2016 02:28
My implementation is lacking, I’ve been slowly moving it up the inheritance heirarchy and believe my goals would require me to implement a StateMachineExecutor.
I want to be able to do two things. First, be able to provide the state machine context (sessions, etc…) to the user defined state machine transition functions (so they can register events). And second, I want to be able to take log compaction into my own hands (even if its just for user-generated Commands). The eventual goal is to allow stepping throug Command history for things like debugging or testing bug fixes.
If anyone has any input, ideas or concerns I would love to hear them. Again thanks!
Jordan Halterman
@kuujo
Jul 01 2016 08:04
That's awesome. If there's missing flexibility we can certainly add it. I can see a need for a different implementation of StateMachineExecutor and StateMachineContext. Atomix has its own implementations that wrap the Copycat provided ones to multiplex state machines
Raghav Babu Subramanian
@RaghavBabu
Jul 01 2016 14:01
Hello,
I was researching on Copycat and Atomix.. And was experimenting their features. Can someone please explain me the major difference between them?? I see both are implementing a fault tolerant Raft consensus algorithm. But I couldn't find any major difference. Please enlighten me.
Roman Pearah
@neverfox
Jul 01 2016 14:04
@gardnervickers Hey! You're into Atomix too?!
We're using it with our Onyx jobs
Gardner Vickers
@gardnervickers
Jul 01 2016 14:05
@neverfox Hey! Neat! I would love to hear more about that
@RaghavBabu From my understanding, Copycat is a distributed state machine based an implementation of Raft, Atomix builds some useful primitives ontop of this like locks, collections, and group membership
Raghav Babu Subramanian
@RaghavBabu
Jul 01 2016 14:19
@gardnervickers Thank you. So you mean, Copy cat is the base, atomix is built on top of copycat to add additional functionality. Its a separate upgradement project.
Gardner Vickers
@gardnervickers
Jul 01 2016 14:32
Essentially yea
Jordan Halterman
@kuujo
Jul 01 2016 20:05
Indeed that is the case. Atomix is really quite simple. Copycat is where all the complexity lies in the Raft implementation. Atomix just extends Copycat's state machines to allow multiple state machines to be multiplexed on a single log with a nicer API. Atomix resources are just Copycat state machines, and in fact they're actually tested directly on Copycat. Atomix just provides the ability to create, modify and delete multiple state machines in a single Copycat cluster. When you create a resource (distributed object) in Atomix, a new Copycat state machine is created in the cluster and a nice API wraps the CopycatClient for communicating with that state machine.
Max Lord
@maxl0rd
Jul 01 2016 21:15
Hi atomix people. I've started building something using groups and direct messaging with request/reply between group members. I've got code that works great in a unit test with several members communicating on one node, but I'm having some trouble getting it to work in production on two nodes. I'm always getting a MessageFailedException in this case.
I'm using 1.0.0-rc7. Any good examples of this floating around that I might be able to crib from?
Jordan Halterman
@kuujo
Jul 01 2016 21:47
https://github.com/atomix/atomix/blob/master/groups/src/main/java/io/atomix/group/messaging/internal/AbstractMessageProducer.java#L67-L79
That exception is thrown when the client expects an ack from the receiving side of the connection but receives a failed acknowledgement due either to an explicit failure or implicit failure
but there could be a bug here
what are the producer options?
Max Lord
@maxl0rd
Jul 01 2016 22:08
Im using direct and request reply. On my last test it seems like the message was actually not received.
Max Lord
@maxl0rd
Jul 01 2016 22:13
I have some code that logs request and response out, and this works in my test case. So im thinking the consumer names etc are correct. Also, it definately joins the group all right. But the message send seems to fail right away and the listener never logs receipt.
Roman Pearah
@neverfox
Jul 01 2016 22:25
@gardnervickers Basically, we're using it to skip processing that other peers may have recently processed (DistributedSet) and for status reports (DistributedValue)
Gardner Vickers
@gardnervickers
Jul 01 2016 22:26
Ah are your peers triggering side effects?
Roman Pearah
@neverfox
Jul 01 2016 22:27
So creating and closing clients via lifecycles and writing on lifecycles and triggers
Yes, it was just a more natural and effective method than ZK, which could have served the purpose but would have required more tricks
Still, it's a bit tricky to mix systems like that but there was a need for a sort of in-memory, distributed KV store to get what we needed.
We needed something Avout-like
so it changes the overall behavior of the job dynamically
Gardner Vickers
@gardnervickers
Jul 01 2016 22:30
Yea gotcha
Roman Pearah
@neverfox
Jul 01 2016 22:31
Atomix is just ridiculously easy to use and being JVM/Java 8 it doesn't have the translation issues. It lives in clojure pretty seamlessly
You can just deref as needed
What got your attention?
Jordan Halterman
@kuujo
Jul 01 2016 22:39
@maxl0rd I'm actually doing some work with those exact features tonight - leader election with members forwarding messages to the leader - so I'll try to reproduce it. The group messaging code is the youngest code in Atomix. Will have to look at logs for a reproducer. What I would be looking for is what the cluster sent back to the client and why. The client should be getting back a GroupCommands.Ack message that caused the exception.
Atomix is just ridiculously easy to use...
i would say say that is the primary design philosophy
Max Lord
@maxl0rd
Jul 01 2016 23:30
Cool. I noticed that the messaging stuff changed with the recent version. I should create a more isolated example so I can share the code.
Jordan Halterman
@kuujo
Jul 01 2016 23:46
Yep. That would be awesome. There should be no more significant changes aside from some serialization API improvements before it's released