These are chat archives for atomix/atomix
I’ve been working on https://github.com/atomix/atomix-test. I still need to document it, but it’s pretty fun to play with.
Set up a 3 node cluster with a consensus configuration:
atomix-test cluster foo setup consensus -n 3
Add a stateless node to the cluster
atomix-test cluster foo add-node client
Tear down the cluster
atomix-test cluster foo teardown
The script can take any configuration file and will set up a Docker network/nodes to form a cluster. It also can be used to mess with the cluster - kill nodes and cause partitions and what not.
java.util.concurrent.CompletionException: io.atomix.primitive.PrimitiveException$Unavailable: Failed to reach consensus
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:769) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) at java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:561) at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:929) at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
Caused by: io.atomix.primitive.PrimitiveException$Unavailable: Failed to reach consensus
... 7 more
@digevil can you copy the node configuration?
One thing you have to be careful of with Raft is that you delete the data directory if you’re trying to start a new fresh cluster. Sometimes when changing versions or even just configurations, the cluster will pick up the old configuration and/or logs from the data directory and behave erratically because of it. This seems like pretty erratic behavior because it implies a Raft session was successfully created but the leader wasn’t able to commit a write.
The data directory is configured in
withDataDorectory in code or the
data-directory key in configuration files for the Raft partition group.
System.getProperty(“user.dir”) + “/.data”or something
consistencyConfig.setRaftId("raft"); consistencyConfig.setRaftPartition(1); consistencyConfig.setRaftGroup(Sets.newHashSet("node1")); consistencyConfig.setRaftStorageLevel(StorageLevel.MEMORY.name());
getDocumentTree? There was an exception in there that was fixed in think in rc4
Stringname. In other words
atomicMapBuilder(“foo”).build()return instances of the same distributed map, but the getter loads the configuration from configuration files and creates only a single client instance per node, while the builder creates an instance on every
build()call where each instance just has a new set of threads and appears to replicated state machines as a distinct session.
No you should use one or the other. The getters create one Java object and one session for each primitive. The builders create one for each
build() call. Using a builder and then getter would create two instances - one distinct instance via the builder and then the singleton getter instance.
If you need to programmatically configure a primitive, you have to use a builder. If you have a large multithreaded application (which we do) then you can also benefit from using builders to create many instances since more instances of a primitive means more parallelism. But if you just need a single instance with strong consistency guarantees then either build one object or use getters if you’re able to configure the primitive in configuration files.
The biggest benefit to getters is just simplicity and to builders is customizability and parallelism. Each instance of a primitive has a separate logical thread pool and orders requests/responses/events independently of other primitives.
The best way to start is probably to use getters to get/create a primitive.
AtomicMap<String, String> map = atomix.<String, String>atomicMapBuilder(“foo”) .withProtocol(MultiRaftPotocol.builder() ... .build()) .build();
DocumentTreeoperation failed? This is a warning in a Raft state machine logging an exception, but it should have been propagated back to the client and caused an exception there. There are no known bugs in any of the primitives in rc3 or later. Wondering if there’s an
AtomicDocumentTreemethod that’s not tested.
Sometimes state machines may also throw exceptions that are handled by the client. What matters is what the client sees.
This should be reproducible with the code that caused it.