These are chat archives for atomix/atomix
MulticastDiscoveryProvider. In my opinion, new node should send a msg TO somewhere, and one or more exist node should ACCEPT the msg. I'm confused cause I cannot find such description like a server-client in document or in the
BootstrapDiscoveryProvideryou’re providing a list of nodes to notify
BootstrapDiscoveryProviderat the same time ?
MulticastDiscoveryProviderthen the multicast protocol is used to notify other nodes of the new node’s existence. Once those nodes learn of a new node, they connect back to the new node over TCP and tell it about the rest of the cluster.
BootstrapDiscoveryProvider, the new node connects to the bootstrap nodes and sends a message telling them about it, then those nodes send a message back telling the new node about all the other nodes.
i'm still confused.. cause I think as a new node, I need a place that I can config to whom I can register myself
for example, I config Server1 as the broadcast node, and new node would register itself to Server1, and Server1 would introduce the new node to the cluster.
let's say like this:
if I have a running cluster : 192.168.10.1:1111 , 192.168.10.2:1111
and a new node 192.168.10.3:1111 want to join the cluster
what shoud I do to the existing nodes and new nodes?
MulticastDiscoveryProvideruse the broadcast of the network (like 192.168.10.255) to communicate to each other?
MulticastDiscoveryProviderthen they will discover each other over multicast. You’re configuring a multicast group instead of specific node information, and the multicast protocol is responsible for nodes finding each other. A joining node broadcasts its information to a multicast group rather than any specific node, and the other nodes listen for join messages sent to that multicast group.
BootstrapDiscoveryProviderthen you are configuring specific nodes to initially join when you construct the provider via
Addresses of the nodes that are already running, i.e. 192.168.10.1:1111 and 192.168.10.2:1111
MulticastDiscoveryProvider, what does
withMulticastAddress("126.96.36.199:54321")mean? does it mean 'I am 188.8.131.52:54321' ?
MulticastDiscoveryProvider, I still need to specify members to use
RaftPartitionGroup. Is there a way to automatically discover the member? Further more, is there a good way to build a cluster with nodes added and removed frequently?
@zwillim these are good questions.
The Raft partition group requires member IDs to be provided because it’s a requirement of consensus. In order to safely bootstrap a consensus (Raft) cluster, the members of the cluster must be static and known by all nodes. So, this can unfortunately limit the usefulness of dynamic clustering. But here’s why this is so...
One of the primary goals of consensus is to ensure a single view of the system’s state especially during network partitions. No matter what node or network failure occurs, Raft will maintain a single copy of the system’s state. Once an operation has been completed on a Raft partition, it’s guaranteed to be successful and persisted forever, no matter what happens to the cluster.
In order to provide this guarantee and avoid split brain - when two portions of the cluster diverge from one another - all the Raft partitions must know which other nodes are participating in the cluster. If they don’t know which other nodes participate in the cluster, they have no way to count votes or determine who won an election.
For example, imagine you start a cluster of three Raft nodes and there’s a network partition. Two nodes are on one side of the partition and one is on the other. The single node sees only itself in the cluster, so it starts and wins an election by voting for itself. The other two nodes see each other and start an election and one of them wins. You now have two leaders in the first term, i.e. split brain. Writes will succeed to either side of the partition, and when the partition heals the nodes will find each other and ultimately elect a single leader. Even after a single leader is elected, you’ll see different views of the system’s state depending on which node you’re reading, and eventually one of the sides of the partition will be overwritten by the other.
So, this is what we can get when Raft nodes try to dynamically discover each other. But actually we can get the same level of consistency using much more efficient algorithms (e.g. the primary-backup algorithm). So, if dynamic clustering for the entire cluster is truly desirable, I’d suggest using the primary-backup algorithm. It will be faster and provide the same guarantees a dynamic Raft cluster would.
But that doesn’t make dynamic discovery useless in the context of Raft. Only nodes that participate in the Raft protocol need to know about each other. Other “client” nodes don’t have to know about the Raft nodes. This is effectively what we do in our application of Atomix: setup a cluster of Atomix agents configured with a Raft partition group, then configure client nodes to discover the Raft nodes and where the Raft partitions live. The client nodes then discovery each other through the Raft nodes, and we maintain the strong consistency guarantees of Raft.