These are chat archives for atomix/atomix

12th
Jan 2018
Jordan Halterman
@kuujo
Jan 12 2018 23:25
yeah I’ve been having issues with the failure detector today too
@vvstej it’s the same as ZooKeeper: all data is held in memory and data in Raft partitions is backed by disk. So, the limitation is memory, assuming I guess that your memory is smaller than your disk :-P
Tej
@vvstej
Jan 12 2018 23:28
gotcha, but if I configure the storage level as StorageLevel.DISK, then its always the disk ,isnt it?
Jordan Halterman
@kuujo
Jan 12 2018 23:31

the StorageLevel is for the Raft logs - DISK/MAPPED/MEMORY will store logs on disk, memory mapped file, and in memory. But that’s just the commit logs. The actual state of the system is held in memory in a set of state machines backed by those logs. So, e.g. if you create a ConsistentMap, the actual Map object that’s replicated is in memory regardless of the StorageLevel. So a read on the map will be read from a Map object in memory.

Of course, data is partitioned so there’s some scalability beyond the memory of a single node. But that just depends on the specific configuration of a cluster and the types of primitives used:
https://github.com/atomix/atomix/blob/master/core/src/main/java/io/atomix/core/map/impl/ConsistentMapService.java#L116

Tej
@vvstej
Jan 12 2018 23:33
Does this apply for the older DistributedMap as well, I am currently using 1+ API version
Jordan Halterman
@kuujo
Jan 12 2018 23:34
More so… new Atomix has partitioning and also supports a primary-backup/multi-primary protocol that replicates primitives more like Hazelcast and thus can scale much better
Tej
@vvstej
Jan 12 2018 23:37
I see, so say hypothetically all machines in the cluster go down, will the data be lost after bringing them back up or is the state machine built from the commit logs? Sorry if this is a rudimentary question.