These are chat archives for atomix/atomix

12th
Nov 2017
Jordan Halterman
@kuujo
Nov 12 2017 09:04

@pawel-kaminski-krk That's right, currently state has to fit in memory but state changes are stored on disk.

The partition module is what does sharding. Each partition is a Raft cluster with its own leader, so there are multiple leaders within the cluster with each distinct state change going through a single leader. For example, PartitionedAsyncConsistentMap.

Atomix 2.1 is nearing completion now. All the features - cluster management, messaging, events, primitives, partitioning, REST API, and CLI - have all been just about completed. One last change to allow custom primitives is on its way, then there will be some sort of release in the near future.

I updated the README with some information and examples for all of the features. The documentation is pretty useless right now, but it is just intended to give an overview of all the work that has gone into Atomix 2. The website will be updated with all new documentation over the next few weeks, and I'll be making some screencasts using the CLI to demonstrate features as well.

Johno Crawford
@johnou
Nov 12 2017 09:16
@kuujo I know we discussed it awhile back now but perhaps it has changed in the meantime, wrt the threading model, if I use say a distributed map and inside the returned future invoke a blocking io operation eg. Db lookup that would disrupt atomix from operating correct? Instead one should consider the async methods provided by cf with a custom executor
Paweł Kamiński
@pawel-kaminski-krk
Nov 12 2017 09:16
@kuujo I stayed tuned
Jordan Halterman
@kuujo
Nov 12 2017 09:25
@johnou yeah that's correct. Atomix does detect blocked futures, but it doesn't detect arbitrary blocked threads inside future callbacks. So, if a future returned by Atomix is join()ed inside another future's callback, that won't prevent any other future's from being completed and won't cause a deadlock. It will just change the order/thread on which futures are completed. But arbitrary blocking inside a future callback could cause Atomix to avoid completing other futures until that callback is unblocked unless is *Async. It always tries to complete futures for a given primitive in the order in which they were created.
...and on a single logical thread
The thread model in the Raft clients is actually configurable though. It can be configured to use an unordered thread pool instead. But last time we tried that we got a memory leak for some reason, so it's currently buggy.
Anyways: The latest Atomix feature is an interactive command line interface for interacting with Atomix nodes. It can be used to inspect the state of the cluster, send/receive messages/events, inspect primitive state, and do everything primitives can do (elect leaders, evict leaders, acquire/release locks, set map values, increment counters, etc). It has autocompletion of keys and prints colorized JSON output. It's intended as a learning tool as much as a cluster management tool.
I'll upload a screencast of the cli some time soon
Johno Crawford
@johnou
Nov 12 2017 09:37
:+1:
Jordan Halterman
@kuujo
Nov 12 2017 23:08
Here are some screenshots of the cli:
Cluster membership
Cluster events
Map get/put
Distributed lock
Distributed unlock
Leader election
Leader election
Jordan Halterman
@kuujo
Nov 12 2017 23:56
Atomix 2.1 should be released at the beginning of December... all the documentation will be rewritten by then.