These are chat archives for RBMHTechnology/eventuate

16th
Apr 2017
Martin Krasser
@krasserm
Apr 16 2017 07:30

There is no retry necessary. A simplified view is that Eventuate implements a hybrid of a first generation approach (section 3.1. in the paper) and a vector clock approach (section 3.2). The first generation approach is the only one that provides non-blocking behavior but has the drawback that a large number of events are transmitted redundantly. Eventuate reduces this redundancy by using vector time so that only events are transmitted that are actually in the causal past of the replication target. The causal past of a replication target is tracked with a version vector (which is the merge result of the vector timestamps of all entries in a target log; the actual vector clock the version vector merged with the local time i.e. sequence number of the target log). Selecting only those events for transmission that are not in the causal past of a replication target is the responsibility of replication filters.

A very simple example: let's assume three datacenters A, B and C where A emits an event e1 with vector timestamp (1,0,0). B now tries to pull new events from A and C and will receive e1 from A (and no event from C) and set its version vector to (1,0,0). After B has written e1 to its target log it receives a replication request from C (i.e. C tries to pull events from B) so that C also writes e1 to its target log settings its local version vector to (1,0,0). A subsequent replication request from C to A (i.e. C tries to pull events from A) also contains C's local version vector (1,0,0) in the request so that A can exclude all events that are in the causal past of (1,0,0) which is the case for e1. In this scenario (where all replication channels are running concurrently) we have avoided the redundant transmission of e1 from A to C.

With causality filters the replication network can always make progress because a given replication target can obtain a given event from more than one source (e.g. C could have obtained e1 from A instead of B). It even allows for dynamic changes of the replication network without additionally using special membership change protocols. We could even show that the transmission redundancy is even rather small (< 2%) even under high write and replication loads (being more or less equivalent to the approach taken in section 3.2 in terms of network usage but without using the drawbacks of a blocking algorithm).