These are chat archives for atomix/atomix

12th
Oct 2018
Jordan Halterman
@kuujo
Oct 12 2018 00:57
yeah multicast is a no go on AWS
that’s why it’s disabled in Atomix by default
most networks actually have it disabled by default
but UDP will work anywhere
looks like we’ve hit a 50 minute build limit in Travis and need to figure out how to run tests concurrently
builds on 3.1 are just failing like crazy because of it
maybe it’s best just to ensure all the tests can run concurrently and enable threading or multiprocessing in surefire
Johno Crawford
@johnou
Oct 12 2018 07:28
Reusable primitives would help
Johno Crawford
@johnou
Oct 12 2018 07:54
@kuujo what's the difference between master and 3.1 at this stage?
master for new features and you are trying to stabilise 3.1?
Jordan Halterman
@kuujo
Oct 12 2018 07:56
Yeah it’s backwards. Probably should switch the two
3.1 is new features
Johno Crawford
@johnou
Oct 12 2018 07:57
ah
yeah that's confusing
:P
Jordan Halterman
@kuujo
Oct 12 2018 07:57
Pluggable membership protocol, new Raft log, probably significant primary-backup improvements, new messaging protocol, maybe Raft partition group configuration changes
I may work on a spec for Raft partition group configuration changes next week and then will do one for primary-backup some time after that... before I start having to work on conference talks
Eric Chavez
@echavez
Oct 12 2018 21:37

@kuujo you had previously mentioned that "Each primitive has a distinct Raft session" I took this to mean that if you had a primitive with a unique name that you would continue to use that distinct Raft session, as long as the session had not been closed, every time you used the builder to retrieve the primitive. So a call like the following would reuse the session and only open a new one if none currently existed

AtomicCounter atomicCounter = atomix.atomicCounterBuilder("AtomicCounterName-ABC-123")
    .withProtocol(
        MultiRaftProtocol.builder(DATA_RAFT_PARTITION_GROUP)
            .withMaxRetries(multiRaftProtocolConfig.getMaxRetries())
            .withRetryDelay(multiRaftProtocolConfig.getRetryDelay())
            .withReadConsistency(multiRaftProtocolConfig.getReadConsistency())
            .withCommunicationStrategy(multiRaftProtocolConfig.getCommunicationStrategy())
            .build()
    ).build();

While testing this I think each call to the block above, using the same name, will continue to open new raft sessions. In order to properly close all AtomicCounters with the name "AtomicCounterName-ABC-123" I would need to track each reference and close each one individually instead of calling close on a single instance of AtomcCounter with the name "AtomicCounterName-ABC-123". Is my understanding correct here?

Jordan Halterman
@kuujo
Oct 12 2018 21:57
Each instance of a primitive has a unique Raft session. Using a builder builds a new instance of the primitive, which is what I mean when I say don’t create a new instance every time you want to use it. If you want to reuse a primitive, just create the primitive and use it. Otherwise, there is a multiton version of the API: atomix.getLeaderElection(...) will always return the same instance/session.
Jordan Halterman
@kuujo
Oct 12 2018 22:04
The reason build() returns a new instance is:
• The method name build() suggests a new instance is built
• It allows separate encapsulated portions of the application running on a node to have different sessions, e.g. so one portion of an application can close its instance of a LeaderElection without implicitly affecting other portions of the application.
• It also allows for more concurrency within an application. Each session is limited by sequential ordering for operations in that session. If multiple portions of a large application share a primitive instance/session, they’ll limit each others’ operations because of session ordering guarantees. But creating multiple instances of a primitive on a node removes dependencies between different portions of an application sharing the same distributed state. This has shown to reduce the chance for cascading failures e.g. when a session is expired in large applications.
This is also documented in the Atomix API:
```the instance of the primitive constructed by the returned builder will be distinct and will not share
  • local memory (e.g. cache) with any other instance on this node```
This also allows for different configurations for different instances of a primitive, e.g. creating a read-only instance for one piece of code, and a read-write instance for another piece of code running on the same node
I don’t love the getter methods, though, because they don’t allow programmatic configuration like builders do. One thing we could do is add a get() or buildSingleton() method to the PrimtiveBuilder to return a singleton instance with a shared session
Jordan Halterman
@kuujo
Oct 12 2018 22:14
One thing you can do (which I’ve experimented with) with multiple instances/sessions is create an instance-per-thread to remove the limitation of single-threaded ordering on a blocking primitive in a multi-threaded application
Each thread has its own instance of the primitives with its own ordering which isn’t affected by concurrent operations on other threads
Jordan Halterman
@kuujo
Oct 12 2018 23:04
@echavez