These are chat archives for RBMHTechnology/eventuate

23rd
Feb 2018
gabrielgiussi
@gabrielgiussi
Feb 23 2018 00:04

one could build a network A -> B where only events emitted at A are replicated to B, but events emitted at B are not replicated to A

I haven't tried replication filters so I want to ask, in this case events generated at A are not causaly related to events generated in B? In other words, events generated at A will always have VectorTime(A -> x) and evens generated in B VectorTime(A -> x, B -> y), or events are not replicated from B to A but it still updates the VectorTime on ReplicationReadSuccess from B?

Volker Stampa
@volkerstampa
Feb 23 2018 07:15
The former is actually true, i.e. the VectorClock at A is not updated through events from B (as non are replicated from B to A) and thus the events emitted at A will always have VectorTime(A: x).
gabrielgiussi
@gabrielgiussi
Feb 23 2018 11:19

make the CRDTs maintain the RTM they need internally and let them decide about their causal stability. They could be configured in form of a priori knowledge about the locations they should consider

We could read a property eventuate.partitions.crdtServiceId from application.conf when a new CRDTService is instantiated, this property should contain the list of partitions to consider for the specified crdtService, e.g. eventuate.partitions.awset-service1 = ["A_log1","B_log1"]
Then, we could mantain a RTM per CRDTService using an EventsourcedView that extracts the pair (emitterId,vectorTimestamp) from all the DurableEvent of the EventLog.
You are thinking in something like that?
Note that the eventuate.partitions.crdtServiceId configuration is error prone, if you add an extra location from which the endpoint will never receive updates, stability will never be reached but it has not impact on correctness (i.e. the values of the crdts will remain correct). In the other hand, if you forget to add a location that is part of the EventLog and is not filtered out (like in the case discussed above), then stability could fire a TCStable(VectorTime(A-> x)) when B has not yet seen VectorTime(A-> x) and this could break CRDTs.

Volker Stampa
@volkerstampa
Feb 23 2018 16:16

We could read a property eventuate.partitions.crdtServiceId from application.conf when a new CRDTService is instantiated, this property should contain the list of partitions to consider for the specified crdtService, e.g. eventuate.partitions.awset-service1 = ["A_log1","B_log1"]

Seems to be reasonable to me.

Then, we could mantain a RTM per CRDTService using an EventsourcedView that extracts the pair (emitterId,vectorTimestamp) from all the DurableEvent of the EventLog.

Not sure if this is feasible?!? Replay times of this view might become infinite (at least without using snapshots). I wonder if it is possible to integrate this into the aggregates?

Note that the eventuate.partitions.crdtServiceId configuration is error prone

Indeed that is correct. But Eventuate configuration is already for experts only. You could for example build replication network with cycles over filtered connections which could lead to persistent event loss (see the warning here). Likewise incautious uses of processors could cause similar problems (as they behave similar in that respect to filtered connections).

... then stability could fire a TCStable(VectorTime(A-> x)) when B has not yet seen VectorTime(A-> x) and this could break CRDTs.

At least that is only transient, isn't it? I.e. it could be fixed by fixing the configuration and restarting the service as the persistent log is not really affected, is it?

gabrielgiussi
@gabrielgiussi
Feb 23 2018 17:48

Not sure if this is feasible?!? Replay times of this view might become infinite (at least without using snapshots). I wonder if it is possible to integrate this into the aggregates?

Do you mean create a RTM per CRDT where the EventsourcedView will have the aggregateId = crdt.aggregateId?
Anyway, snapshots should prevent heavy replays, maybe scheduled snapshots?

At least that is only transient, isn't it? I.e. it could be fixed by fixing the configuration and restarting the service as the persistent log is not really affected, is it?

Is not transient if you take a snapshot of the CRDT. The snapshot will persist a wrongly pruned POLog.