These are chat archives for atomix/atomix

2nd
May 2018
Jordan Halterman
@kuujo
May 02 2018 00:28

I’ve been working on simplifying the API used by distributed primitives as I’m writing the documentation. A lot of the process of setting up primitives can be pretty cumbersome, so I refactored the API to use service proxies and make it easier to manage partitioning. Primitives can now supply a service interface:

public interface ConsistentMapService {
  @Operation(value = “containsKey”, type = OperationType.QUERY)
  boolean containsKey(String key);

  @Operation(value = “put”, type = OperationType.COMMAND)
  byte[] put(String key, byte[] value);
}

And then proxies can be written to operate directly on that interface:

public class ConsistentMapProxy extends AbstractAsyncPrimitiveProxy<AsyncConsistentMap<String, byte[]>, ConsistentMapService> implements AsyncConsistentMap<String, byte[]> {
  @Override
  public CompletableFuture<Boolean> containsKey(String key) {
    return invokeBy(key, service -> service.containsKey(key));
  }

  @Override
  public CompletableFuture<byte[]> put(String key, byte[] value) {
    return invokeBy(key, service -> service.put(key, value));
  }
}
The AbstractAsyncPrimitiveProxy class has methods for invoking operations by key, by partition, or on all partitions, and a proxy can be used to convert method calls into raw PrimitiveOperations
the same type of proxies will be available within replicated state machines for calling methods on client-side proxy classes
public class DefaultConsistentMapService extends AbstractPrimitiveService<ConsistentMapClient> implements ConsistentMapService {
  @Override
  public byte[] put(String key, byte[] value) {
    if (map.put(key, value) == null) {
      getCurrentSession().accept(client -> client.onInsert(key, value));
    } else {
      getCurrentSession().accept(client -> client.onUpdate(key, value));
    }
  }
}
Jordan Halterman
@kuujo
May 02 2018 00:35
The use of proxies significantly reduces the amount of primitive code, removing all the operations/events classes. It also inforces a strong interface between client-side proxies and the replicated state machines - a method invoked in a client-side proxy will always exist on the server.
Jordan Halterman
@kuujo
May 02 2018 06:00
keeping these PRs to a manageable size BTW, so going to be a bunch of them incoming
Johno Crawford
@johnou
May 02 2018 06:00
What about complex objects, how is the kryo decoder / encoder setup or does it not support that?
Had to call it a night early yesterday, didn't manage to pinpoint the issue causing instability in the tests yet
Jordan Halterman
@kuujo
May 02 2018 06:03
The serialization API is the one thing I haven’t quite figured out yet. It’s still ugly. Currently, the PrimitiveType just provides a serializer Namespacethat is used to serialize all the operations. It would be possible to register types automatically by reading the service interface, but it would be difficult to maintain the same serializer state across versions for rolling upgrades, so I suspect we just need to deal with some form of explicit type information for serialization… or use class names when rolling upgrades are enabled, which sucks.
Luca Burgazzoli
@lburgazzoli
May 02 2018 06:06
@kuujo I've just opened this issue atomix/atomix#529 but maybe better to talk here
basically in atomix v1 there were the possibility to attach metadata upon registration, now such metadata stuffs is lost
and we have now tags on Member level but it is quite less flexible, any chance to get metadat asupport back ?
Jordan Halterman
@kuujo
May 02 2018 06:08
It may be difficult to store arbitrary objects because of the way serialization works and because we want information to be readable via the REST API and writable in configuration files as well, but we can certainly change the structure of the data attached to Member objects
Johno Crawford
@johnou
May 02 2018 06:30
we have a concept of protocol version in our product which I could talk a little about, need to drop off the kid first though
Luca Burgazzoli
@lburgazzoli
May 02 2018 06:32
a simple Map<String, String> would be enough as metadata at least for my use case :)
Jordan Halterman
@kuujo
May 02 2018 06:55
Yeah I think that makes sense. You want to change it and submit a PR?
Luca Burgazzoli
@lburgazzoli
May 02 2018 07:17
I can try, so should this replace tags ?
Jordan Halterman
@kuujo
May 02 2018 07:18
yep
Luca Burgazzoli
@lburgazzoli
May 02 2018 07:18
ok let me try
Jordan Halterman
@kuujo
May 02 2018 07:18
Awesome! Love new contributions :-)
Johno Crawford
@johnou
May 02 2018 08:03
@kuujo still up?
Luca Burgazzoli
@lburgazzoli
May 02 2018 10:28
another question, when building a "client" why is it required to give an address ?
Johno Crawford
@johnou
May 02 2018 12:06
https://help.github.com/articles/setting-up-an-apex-domain/ looks like you might just need to update the a addresses for https support
Johno Crawford
@johnou
May 02 2018 18:36
@kuujo can you check out atomix/atomix#503 please? I think it might help with some issues I'm seeing locally
wrt versioning, we have something like this
enum ClientProtocolVersion {

    DEVELOPMENT(0, null, true),

    VERSION46(46, "2.25"),
    VERSION47(47, "2.26"),
    ;

    public boolean compatibleWith(ClientProtocolVersion protocolVersion) {
        if (ClientProtocolVersion.DEVELOPMENT.equals(protocolVersion)) {
            throw new IllegalArgumentException("You should not be comparing DEVELOPMENT protocol version!");
        }
        return this.alwaysCompatible || this.version >= protocolVersion.version;
    }
then for specific objects
    @Override
    public void writeTo(ClientProtocolVersion protocolVersion, OutgoingMessage message) {
        message.writeInt(id);
        message.writeBoolean(completed);
        message.writeString(name);
        message.writeBoolean(progressable);
        if (progressable) {
            message.writeInt(currentProgressPoint);
            message.writeInt(targetProgressPoint);
        }

        if (protocolVersion.compatibleWith(ClientProtocolVersion.VERSION50)) {
            message.writeString(templateName);
        }
    }
not sure if we can apply something like this to kryo, or use the compatible serializer that you mentioned in a talk.. no experience with it though
Jordan Halterman
@kuujo
May 02 2018 19:44
Well, we need to be able to do rolling upgrades, which means different nodes can be running different versions at the same time, and serialized bytes for the same class can differ across versions. This is why we use Kryo’s CompatibleFieldSerializer which writes the field names at the header of the object and then the values. This allows the deserializer to determine which fields are present during deserialization. The limitation is then that fields can’t be renamed or re-typed.
Johno Crawford
@johnou
May 02 2018 19:47
need to put the kid to bed but I was currently looking at the sample in atomix/atomix#531
with my suggestion it still fails to start
io.atomix.core.impl.CoreTransactionService#start never returns for the client
not sure why yet
Jordan Halterman
@kuujo
May 02 2018 19:50
I will try it out
Johno Crawford
@johnou
May 02 2018 20:26
looks like it's trying to continuously send write operations to the client member
something wrong with the profiles?
Jordan Halterman
@kuujo
May 02 2018 20:27
probably partitioning
Johno Crawford
@johnou
May 02 2018 20:56
is a client suppose to have a CoreTransactionService?
Jordan Halterman
@kuujo
May 02 2018 20:56
yes
Johno Crawford
@johnou
May 02 2018 20:57
never ending loop in io.atomix.protocols.backup.proxy.PrimaryBackupProxy#execute(io.atomix.primitive.operation.PrimitiveOperation, io.atomix.utils.concurrent.ComposableFuture<byte[]>)
continues retrying threadContext.schedule(Duration.ofMillis(RETRY_DELAY), () -> execute(operation, future));
Jordan Halterman
@kuujo
May 02 2018 20:59
probably because the client is being included in the primary election, so it expects a primary-backup server to be available on that node
hmm maybe not
ahh yeah I know why
it’s because the HashBasedPrimaryElection is being used since no Raft partition is available to do the election through Raft, and that primary election provider isn’t honoring the partition group membership.
it’s still just asumming a partition group belongs to all nodes
maybe need to add that PartitionGroupMembershipService after all for the eventually consistent primary election
Johno Crawford
@johnou
May 02 2018 21:04
two birds one stone
:P
was atomix/atomix#503 good to merge btw? even with the commitIndex assignment in reset?
i'm not able to merge it until tests start passing again
Jordan Halterman
@kuujo
May 02 2018 21:17
Unfortunately, I wrote that PartitionGroupMembershipService and then ditched it. Gotta rewrite it again
Decided to forgo the ability to reconfigure Raft partitions until post 2.1. There are too many complexities to it to try to deal with it now.
Ugh I hate the Gitter iOS client
Good find though. Need to add tests for all of the configurations.
Jordan Halterman
@kuujo
May 02 2018 23:25
hmm those extra partition groups sure seem to be messing everything up