These are chat archives for atomix/atomix

27th
Apr 2018
Johno Crawford
@johnou
Apr 27 2018 05:43
@rbondar please wait for the next beta / rc
huge changes coming
@kuujo ran into this code smell io.atomix.core.election.impl.LeaderElectionService#promote
      Leadership<byte[]> oldLeadership = leadership();
      if (oldLeadership == null) {
        return false;
      } else {
where leadership() can never return null
Jordan Halterman
@kuujo
Apr 27 2018 06:13
yeah need to go through inspections probably
Jordan Halterman
@kuujo
Apr 27 2018 06:23
The partitioning/protocol change is the last PR. Nothing left on the bucket list for the RC unless we happen to come up with some other change that’s needed.
With all the PRs merged I think it will be time to go through all the code and clean up the types of things like the above and add a lot more tests. Need a couple weeks of that to be able to have an RC.
Jordan Halterman
@kuujo
Apr 27 2018 06:30
Also need to ensure the Python/test framework is updated to add some basic tests that involve container clusters. I did that for the most part recently. Probably just needs a little work and then tests.
@rbondar you should check out the website documentation on cluster configuration: http://atomix.io/docs/latest/user-manual/cluster-management/
that’s based on the latest codes in master, which is what should go into an RC and the final release now

The cluster has to be configured with a management group which is used by all the rest of the partition groups to e.g. do primary election, store primitive information, etc. Then you have to configure primitive partition groups as you did above, but the correct way to do that would be:

builder.addPartitionGroup(PrimaryBackupPartitionGroup.builder(“data”)
  .withNumPartitions(32)
  .build());

Even simpler, though, you can now use profiles to configure the cluster:

Atomix atomix = Atomix.builder()
  .withLocalMember(...)
  .withMembers(...)
  .withProfiles(Profile.DATA_GRID)
  .build();
Jordan Halterman
@kuujo
Apr 27 2018 06:40
Then to create a primitive you have to provide a PrimitiveProtocol instance that’s compatible with the partition group in which you want to store the primitive.
ConsistentMap<String, String> map = atomix.consistentMapBuilder(“my-map”)
  .withProtocol(MultiPrimaryProtocol.builder(“data”)
    .withNumBackups(2)
    .withReplication(Replication.ASYNCHRONOUS)
    .build())
  .build();
Johno Crawford
@johnou
Apr 27 2018 07:27
@kuujo poll methods causing problems?
Jordan Halterman
@kuujo
Apr 27 2018 07:28
Not really compatible with partitioning. Can’t exactly poll all the partitions otherwise you’re removing one entry per partition. Would have to use some sort of lock, but that’s probably work for another time.
Johno Crawford
@johnou
Apr 27 2018 07:29
ok and the difference between that and say, ceilingentry is that it modifies the map?
Jordan Halterman
@kuujo
Apr 27 2018 07:30
yeah
Johno Crawford
@johnou
Apr 27 2018 07:30
alright
just curious, I don't actually use that method
Jordan Halterman
@kuujo
Apr 27 2018 07:32
it’s supposed to remove the lowest/highest entry, but to figure out which partition that's in we have to read them first, so it would require at least two operations on the corrent partition which would cause consistency issues. Between the two operations a new lowest/highest entry may be added.
Johno Crawford
@johnou
Apr 27 2018 07:35
yep
here's a few safe fixes, current master fails to build in intellij because of the osgi bug
atomix/atomix#518
Johno Crawford
@johnou
Apr 27 2018 08:38
Simplify the DocumentTree implementation to use a single partition.
ahhaah
Jordan Halterman
@kuujo
Apr 27 2018 08:48
Ugh now my internet isn’t even working 🤷‍♀️
Well that’s done. Will review the rest of the PRs tomorrow
DocumentTree really needs to be redesigned for partitioning. It has never worked well
Johno Crawford
@johnou
Apr 27 2018 08:48
must be a sign to goto bed
don't want keyboard face two nights in a row
Jordan Halterman
@kuujo
Apr 27 2018 08:49
New Javadoc is up on the website too: http://atomix.io/docs/latest/api/
Johno Crawford
@johnou
Apr 27 2018 08:49
it might stay that way
Jordan Halterman
@kuujo
Apr 27 2018 08:49
Gotta filter out a lot more of those classes
There’s a bunch of requests/responses and utilities in there
Ugh my son is sick and awake again anyways adios
Johno Crawford
@johnou
Apr 27 2018 08:50
hope he gets better soon :+1:
Jordan Halterman
@kuujo
Apr 27 2018 17:54
I think there’s just one more test failing ATM
after #519
Jordan Halterman
@kuujo
Apr 27 2018 19:06
I think they should pass after the transaction changes are rebased
Jordan Halterman
@kuujo
Apr 27 2018 21:46
@johnou FYI you have write access to the repo now
Jon Hall
@jhall11
Apr 27 2018 21:47
with great power comes great fear of doing something stupid
or something like that
Johno Crawford
@johnou
Apr 27 2018 23:32
cheers
Jordan Halterman
@kuujo
Apr 27 2018 23:35
feel free to pollute the repo with new branches :-)
that you don’t touch for six months and then eventually delete
that’s my strategy anyways
Johno Crawford
@johnou
Apr 27 2018 23:36
prototyping ftw
Jordan Halterman
@kuujo
Apr 27 2018 23:36
eventually I just forget why I created the branch and suspect I can just delete it
will get all these PRs closed again some time...
Johno Crawford
@johnou
Apr 27 2018 23:41
i'm just rebasing that backup one now
Johno Crawford
@johnou
Apr 27 2018 23:51
just read your comments on the original pr though
Jordan Halterman
@kuujo
Apr 27 2018 23:51
gonna need a little back and forth
pretty interesting question
I seem to recall you mentioning that rebalancing doesn't exist yet though?
or maybe resizing a partition