These are chat archives for atomix/atomix

14th
Apr 2018
Daniel Tam
@dualscyther
Apr 14 2018 01:57
Apologies, I know I could look through the source for this, but does Copycat maintain permanent tcp connections to other nodes in the cluster while its up and running? I assume the answer is yes for performance reasons
Jordan Halterman
@kuujo
Apr 14 2018 03:52
$15 for internet that didn’t even work on that damn flight

@dualscyther Copycat maintains a single TCP connection to all other nodes, and clients have connections to all servers.

In Atomix all nodes - including clients/servers - are peers and have both TCP servers and TCP connection pools (up to 8 connections) to each other peer. Messages with the same subject are always sent on the same TCP connection to maintain causal order.

Daniel Tam
@dualscyther
Apr 14 2018 05:21
so each peer has a connection pool to each other peer. And each connection pool has up to 8 simultaneous connections?
Jordan Halterman
@kuujo
Apr 14 2018 07:37
Indeed
Daniel Tam
@dualscyther
Apr 14 2018 08:05
I assume for performance reasons there's always a connection kept open between each peer to reduce the handshaking overhead?
Jordan Halterman
@kuujo
Apr 14 2018 09:10

Yeah all 8 connections to all other nodes are kept open. The pool is shared by messages of all types, and message types are hashed to specific connections.

Although, we may want to at least limit CLIENT nodes to a single connection per DATA/CORE node and reserve pools only for nodes that have to replicate data.

@johnou wdyt?
This can be added to the NettyMessagingService builder
Pool size or a pool boolean
Johno Crawford
@johnou
Apr 14 2018 09:42
As long as there is no blocking should be ok
Was also trying to think of live lock scenarios
Jordan Halterman
@kuujo
Apr 14 2018 09:44
My thought is just that the number of client nodes can be drastically greater than the number of data/core nodes, and the number of connections will grow exponentially as it is
Johno Crawford
@johnou
Apr 14 2018 09:46
Yeah depends on the application but I do see that as a problem in some implementations
Might be interesting to benchmark
Jordan Halterman
@kuujo
Apr 14 2018 09:47
Yep
Ugh I gotta get to sleep. Been traveling all day
Johno Crawford
@johnou
Apr 14 2018 10:53
Hit me up if you're ever in Italy
Daniel Tam
@dualscyther
Apr 14 2018 14:46
message types are hashed to specific connections
not sure I understand this part
but yeah ok cool that makes sense
thanks!
Jordan Halterman
@kuujo
Apr 14 2018 18:45

@dualscyther look at the API:
https://github.com/atomix/atomix/blob/master/messaging/src/main/java/io/atomix/messaging/MessagingService.java

sendAndReceive(Address address, String type, byte[] message);

In specific usages of the API, a message type will be e.g. cluster-heartbeat or command in the Raft implementation or execute in the primary-backup implementation. Generally, a specific type of object is associated with the String type, e.g. HeartbeatMessage or CommandRequest or ExecuteRequest. That String type is hashed to a TCP connection which guarantees that it will be delivered in order among other messages of the same type. This reduces the frequency of having to reorder requests inside Raft servers, and the primary-backup implementation actually doesn’t reorder messages at all but instead relies upon those semantics to ensure ordering of requests/events.

@johnou I actually have a lot of co-workers in Italy. Haven’t managed to make it that far from California yet, but 🤞
Last year our conference was in South Korea the week my baby was due so I didn’t even get to go there :-(
Johno Crawford
@johnou
Apr 14 2018 20:21
Oh man, well I haven't been there myself but I'm sure it would have been interesting.
Any co-workers in the Venice area?
Jordan Halterman
@kuujo
Apr 14 2018 22:17
Not sure where in Italy they live