These are chat archives for atomix/atomix

3rd
Nov 2016
terrytan
@txm119161336_twitter
Nov 03 2016 02:42
@kuujo currently ,the major compaction and minor compaction are running respectively, which means ,by coincidently, maybe they will run at the same time, there is a step by which you replace the
the segments list with the same index ,like segmentlist.put(); if the minor compaction do this step after the major compaction(although you have sync keyword,),
maybe the major compacted segment will be discarded.
Mark Elliot
@markelliot
Nov 03 2016 19:31

I'd like to set up a service that can dole out monotonically increasing numbers. This seems trivial to build on top of atomix using a DistributedLong, but I'm worried about overall throughput of the DistributedLong since I'm planning to request new numbers from this service with very high frequency. My existing implementation uses a leader-aware filter to accept requests only on one replica (that is the leader) and to hand out numbers only from the leader -- I check leadership before responding to a request. Since I want my numbers to be persistent, I also request a pool of numbers to hand out (by setting a new ceiling in my persistence layer), so I can hand out those numbers quickly.

It's not too hard to set up a similar leader-aware-filter that wraps my DistributedLong, and then use the DistributedLong to capture the ceiling of values I care about by listening for an election and determining if a particular replica is in fact the leader of the group, and turning on/off the filter.

This gives rise to a couple of questions I'm hoping you might be able to answer/point me to docs on:

  • what's the expected throughput/latency for a DistributedLong for a size 3 cluster in the same data center?
  • is there a way to easily (and cheaply) check that a leader is still the leader?
  • what kind of delay/guarantees are there around when elections run, given, say, the leader ends up in a lone network partition?
Mark Elliot
@markelliot
Nov 03 2016 19:41
Maybe not super clear from my description: my fear is that if the leader becomes segregated it can still issue fresh numbers (from its allowed buffer, up to the ceiling) and that a leader election could succeed while the node that's become segregated has not received an event indicating it is no longer the leader.