These are chat archives for atomix/atomix

20th
Apr 2018
Jordan Halterman
@kuujo
Apr 20 2018 01:14
new Atomix website is up!
soon to be populated with actual documentation other than the getting started guide :-)
Jordan Halterman
@kuujo
Apr 20 2018 01:30
Atomix is done! Please hault all work ;-)
Jordan Halterman
@kuujo
Apr 20 2018 07:53
Atomix 2.1.0-beta3 has been released.
The next release will be 2.1.0-rc1 once the documentation is more complete and the API seems stable
Jordan Halterman
@kuujo
Apr 20 2018 08:09
Hmm... what if we rename CORE and DATA nodes to STATIC and DYNAMIC or PERSISTENT and EPHEMERAL. This is looking at cluster management as independent from storage/replication. The requirements of Raft are that the membership not dynamically evolve, so we just need node types that meet those requirements. STATIC/PERSISTENT nodes would always exist in the configuration whether dead or alive until removed, and DYNAMIC/EPHEMERAL nodes are either alive or not members of the cluster.
Seems more obvious what their behavior is
Johno Crawford
@johnou
Apr 20 2018 08:10
I like both of those over core and data
Jordan Halterman
@kuujo
Apr 20 2018 08:10
Raft partitions may just only be assigned to STATIC nodes
Johno Crawford
@johnou
Apr 20 2018 08:10
to me core means it's required
not optional
Jordan Halterman
@kuujo
Apr 20 2018 08:10
Yep
Will sleep on it and choose one tomorrow
Johno Crawford
@johnou
Apr 20 2018 08:12
a5d5841e-e607-44fc-b6f1-7fcaca9cdea8.jpeg
Jordan Halterman
@kuujo
Apr 20 2018 08:13
lol
Spot on
Jordan Halterman
@kuujo
Apr 20 2018 08:22

Also would like to decouple the system partition from the node types as well and perhaps allow a system partition/group to be provided.

With that done, I also think it could be useful to have some sort of abstraction that provides common configurations. My concern is that the decoupling of the protocols from primitives makes it more complicated to configure Atomix and its primitives, and it would be nice to have a high level abstraction that provides e.g. an enum or something for common configurations (i.e. a configuration for large scale data, for f+1 fault tolerance, for strong consistency/coordination, etc and consistency/persistence/performance based configurations for primitives).

Not sure what that would look like yet
Something that provides a high level API that doesn’t require understanding distributed systems protocols to get the desired behavior
and guarantees
:thinking_face: where are you!?
Off to bed
I hope...
Johno Crawford
@johnou
Apr 20 2018 08:25
ahahah
me or your face?
off to work for me now
Jordan Halterman
@kuujo
Apr 20 2018 08:26
The thinking face emoji!
🤔
There we go
Too used to Slack
Johno Crawford
@johnou
Apr 20 2018 08:27
maybe profiles or something for quick cluster setup
Jordan Halterman
@kuujo
Apr 20 2018 08:27
Yes that’s the term I’m looking for
Johno Crawford
@johnou
Apr 20 2018 08:27
and custom profiles for low level configuration of primitives
Jordan Halterman
@kuujo
Apr 20 2018 08:28
Something that makes getting started much simpler
Jordan Halterman
@kuujo
Apr 20 2018 08:34
Maybe cluster profiles and primitive prototypes
That’s sure to keep me awake for a while :-P
Johno Crawford
@johnou
Apr 20 2018 08:35
sorry :worried:
Jordan Halterman
@kuujo
Apr 20 2018 08:40
At least we got the naming down... that’s always the hard part :-P
Alex
@AtomAlex
Apr 20 2018 09:39
@kuujo thanks for your reply. i got it working now. took me a while to figure out how services, primitives and proxys exactly interact but i got it :smile: great to see you making progress with this project!
rbondar
@rbondar
Apr 20 2018 16:14
@kuujo just moved from 2.1.0-beta2 to 2.1.0-beta3 , removed obsolete "Endpoint" imports and usages, compiled everything fine, started services and found in logs Unknown partition group type: multi-primary io.atomix.utils.ConfigurationException: Unknown partition group type: multi-primary . Have not found answer in currently cut "getting started section" on website.
rbondar
@rbondar
Apr 20 2018 16:37
my bad, its two more dependencies required
do we still need to use builder.withDataDirectory(dataDirectorypath)? PartitionGroups.getPrimaryBackupGroupFactory().createSystemGroup(3,dataDirectorypath) is almost the same ?
Jon Hall
@jhall11
Apr 20 2018 17:28
re STATIC and DYNAMIC vs PERSISTENT and EPHEMERAL, The latter makes me think more about the storage persistance rather than cluster membership, so from that perspective, I like STATIC and DYNAMIC more.
Johno Crawford
@johnou
Apr 20 2018 17:29
@jhall11 sgtm
@jhalterman 3/4
Jon Hall
@jhall11
Apr 20 2018 17:32
I did see a cat at first for the Rest icon, so maybe @jhalterman has a point
Johno Crawford
@johnou
Apr 20 2018 17:50
232773-200.png
junzhai
@junzhai
Apr 20 2018 17:59
test
Hi Guys, Anyone know what's going on with http://atomix.io/? Can't see the old docs...
Johno Crawford
@johnou
Apr 20 2018 18:00
preparing for atomix 2.1.0-rc
junzhai
@junzhai
Apr 20 2018 18:01
thx
@kuujo might want to update the logo here too https://github.com/atomix
Jordan Halterman
@kuujo
Apr 20 2018 20:12
oh yeah thanks
I think I’m favoring PERSISTENT/EPHEMERAL partly because that is how they behave in terms of state as well
Johno Crawford
@johnou
Apr 20 2018 20:14
Or COORDINATION_DATA and DATA?
Johno Crawford
@johnou
Apr 20 2018 20:23
Because they both store data, just one provides extra functionality (Raft)
Or CONSISTENT_DATA and DATA
Johno Crawford
@johnou
Apr 20 2018 20:30
CONSENSUS_DATA
Johno Crawford
@johnou
Apr 20 2018 21:39
@kuujo wdyt
junzhai
@junzhai
Apr 20 2018 22:40
Is there a way to do a distributed read/write lock with fairness setting like java's ReentrantReadWriteLock(true)? thx
Jordan Halterman
@kuujo
Apr 20 2018 22:53
Been working my second job as a firefighter today
@junzhai the DistributedLock will be granted to the longest waiting process, but we don’t currently have a read/write lock. I always thought that would be cool to have. Just need to add a new primitive for it.
Jordan Halterman
@kuujo
Apr 20 2018 22:59
@johnou nodes don’t necessarily have to store data, though, which is why I wanted to move away from the data-centric names. Persistent/ephemeral just happens to apply to data which may make them easier to understand with respect to storing data, but the true meaning is that the nodes are persistent or ephemeral in the cluster configuration, not with respect to data. It should perhaps be feasible to run a cluster without any data storage. That also makes me think we could also get rid of CLIENT nodes as well.
I’d probably keep CLIENT as it’s simpler to understand, though. Clients are really a special type of EPHEMERAL node that can’t be assigned partitions.