These are chat archives for RBMHTechnology/eventuate

13th
Dec 2016
Fabian Koehler
@fkoehler
Dec 13 2016 09:17
Do I understand this correctly that when I use the ORCartService for CRDTs that I am „stuck“ with the sample adding of quantities to items keys? I thought about using the service for a wishlist which I would also give a name and assign it to a user and be able to add items to it. To me it looks like if I want to do that I would need to implement my own CRDT service or revert back to a normal implementation with EventsourcedActors?
Martin Krasser
@krasserm
Dec 13 2016 10:20
@fkoehler if you want to model a wishlist for a user, create an ORSet per user and add the wishlist tems to it. If you additionally need quantities for the wishlist items, use an ORCart per user. Does that help?
Fabian Koehler
@fkoehler
Dec 13 2016 12:48

@krasserm: Into an ORSetService[A]I can put things of type T, the wishlist items as you suggested. But in addition I would like to save more metadata for a wishlist like: userId it belongs, is it public or not and a name for it. To me it seems I want an aggregate like this:

case class WishlistItem(variantId: VariantId, name: String)
 case class Wishlist(userId: String, name: String, items: Vector[WishlistItem])

As far as I understand the ORSet I can not model this ?!

I’ve got another question regarding deletion of events. We also want to persist wishlist items for anonymous users which might get turned into real ones for a logged in user. That means that I would collect wishlists for anonymous users which might get obsolete i.e. after the session times out. Looking at http://rbmhtechnology.github.io/eventuate/reference/event-log.html#deleting-events it’s not possible to delete those aggregates from the log when I use the C* backend. Does that mean I need to keep them forever?

Maybe I do not use EventSourcing for those aggregates and just persist them in the database or I live with those obsolete events which might be actually fine but depends on usage, etc. Not sure yet.

gabrielgiussi
@gabrielgiussi
Dec 13 2016 12:54
I think you could make an EventsourcedActor for your userId and name that holds a reference to the ORSetService and keep a relation between the aggregateId of the EventsourcedActor and the ORSet. I've done this already.
Fabian Koehler
@fkoehler
Dec 13 2016 12:56
That sounds possible but it sounds like you do not have a proper aggregate like the one I mentioned above?
gabrielgiussi
@gabrielgiussi
Dec 13 2016 13:00
As I see it, is an aggregate composed by an EventsourcedActor and an ORSet (the ORSet instance should only be accesed by the EventsourcedActor and not directly via invocations to the service). This is the only way I can think of to compose CRDT for now, maybe there is a better way.
Fabian Koehler
@fkoehler
Dec 13 2016 13:01
Ah I see. I think I might revert back to a non CRDT then as I would prefer to have a proper aggregate case class
gabrielgiussi
@gabrielgiussi
Dec 13 2016 13:08
You should wait to see what the others say, there are more authorized voices in this forum than mine. I just wanted to share what I've been doing this past days.
Why do you want to use a CRDT anyway? What is your use case?
Martin Krasser
@krasserm
Dec 13 2016 13:58

As @gabrielgiussi already mentioned, I'd also model Wishlist separately from the mutable list of items, assuming the metadata are immutable i.e. are only created but never updated. For example

case class Wishlist(userId: String, name: String, itemsCrdtId: String)

defines the wishlist metadata and references the items CRDT by id. A service that wants to update the wishlist for a user should first find the wishlist by userId from a view and then update the CRDT with the obtained itemsCrdtId.

Martin Krasser
@krasserm
Dec 13 2016 14:07
Event deletion is not implemented yet for Cassandra but is planned. Assuming the feature already exists, I'd store events of anonymous users in a separate log that is then cleaned up based on application defined criteria (retention time or whatever ...). I cannot make any promises ATM when we start working on that feature ... (it is a bit more involved as deletions in replicated event logs should be causally consistent).
gabrielgiussi
@gabrielgiussi
Dec 13 2016 14:22
@krasserm What do you think about that Wishlist(id,name,crdtID) that you mention be part of the state of an EventsourcedActor that also contains a reference to the CRDTService. In this case you don't "update the CRDT with the obtained itemsCrdtId" directly but sending commands to the EventsourcedActor that translates to invocations of his service in the event handler.
  1. Client obtains ActorRef of EventsourcedActor in some way
  2. Client sends AddItem to the ActorRef
  3. EventsourcedActor validates command against his state (potentially consuming the value of the CRDT) and persist an event ItemAdded
  4. onEvent { case ItemAdded(item) => orSetService.add(actorAggregateId,item) }
Martin Krasser
@krasserm
Dec 13 2016 14:25
@gabrielgiussi orSetService.add represents a command and should therefore not be executed by an event handler. I know, it's idempotent in this case but still not recommended design.
gabrielgiussi
@gabrielgiussi
Dec 13 2016 14:27
True. So we should invoke the service (aka send command to the CRDTActor) directly in the command handler of the Actor? Or you discard this approach completely?
Martin Krasser
@krasserm
Dec 13 2016 14:32
I think it's not necessary to send every item update through a Wishlist actor. I client should read wishlist metadata only once and then make several updates to the wishlist items CRDT directly, if needed.
You could still provide a service facade to make this more convenient for a caller
gabrielgiussi
@gabrielgiussi
Dec 13 2016 14:48
What if the metadata is mutable? For example if the user should be able to change the Wishlist name. (I'm only using the Wishlist example because was the original example from Fabian, but I'm thinking in aggregates like the ones we implement with EventsourcedActor wich a part of his state is a CRDT).
Maybe I cant ask the question as: "Can be a CRDT part of a EventsourcedActor's state?"
Following the example, maybe the Wishlist contains a WishList type that restricts the kind of items that can be added, to propose a case where the invocation to the service is validated against the state of the actor.
Martin Krasser
@krasserm
Dec 13 2016 16:48

Can be a CRDT part of a EventsourcedActor's state?

Sure, all CRDT services are implemented like that as mentioned in http://krasserm.github.io/2016/10/19/operation-based-crdt-framework/

What if the metadata is mutable

Then consider implementing the whole aggregate as custom CRDT with the CRDT development kit (see above link), or, if you want to work with the lower level actor API you can make CRDTs part of the actor state but then you re-implement logic that is already defined in the abstract CRDTService.

gabrielgiussi
@gabrielgiussi
Dec 13 2016 17:26

consider implementing the whole aggregate as custom CRDT.

Ok, I will give it a try to this approach when I have time, meanwhile I'continue working in an example where I've taken the approach proposed (Actually, I was between the two approaches a weeks ago). I will upload the example to github in a couple of days.

if you want to work with the lower level actor API you can make CRDTs part of the actor state

You mean hold an ActorRef to a CRDTActor in my EventsourcedActor and send it the commands that are currently being send by the service? If I understood correctly this will mean making CRDTActor class public.
_

Martin Krasser
@krasserm
Dec 13 2016 17:47
I rather meant making only the CRDT object part of the actor state and process consumed events by translating them on to CRDT method calls. You could then even use application-specific events (e.g. WishlistItemAdded) instead of the pre-defined operations in the eventuate-crdt module.
gabrielgiussi
@gabrielgiussi
Dec 13 2016 17:49
Now I get it. Thanks for your time Martin.
Fabian Koehler
@fkoehler
Dec 13 2016 17:50
That sounds reasonable. Do the CRDT services have any kind of actor cache? If yes, how does this work with many different requests to the service?
gabrielgiussi
@gabrielgiussi
Dec 13 2016 17:55
Check the CRDTManager inside CRDTService, it contains a Map[String,ActorRef] of actors currently in memory. Invocations to the service are made specifying the id of the CRDT that is traslated to the id of the CRDTActor and then lookuped in the map. There is a relation one to one between CRDTActor and CRDT. The id of the CRDT is something you define like "orSet1" and then the id of the CRDTActor is somethink like "ORSet-orSet1"
Fabian Koehler
@fkoehler
Dec 13 2016 17:58
Ok, that means one would need to manually tear down actors once the Map gets too large.
Martin Krasser
@krasserm
Dec 13 2016 18:25
That's the way to do it for the moment. There should be something that enforces a retention policy. PRs are welcome :smiley: