These are chat archives for RBMHTechnology/eventuate

4th
Aug 2016
Alexander Semenov
@Tvaroh
Aug 04 2016 05:49
Gents, when I'm persisting events (during commands processing), I have some additional information (like correlation id, etc) that I need to access in event handlers but that I don't want to go to event data (it might even be non-serializable). How could I not loose it?
Alexander Semenov
@Tvaroh
Aug 04 2016 06:43
ideally I would like persist methods to accept some additional argument like metainfo and onEvent handler would get some wrapper object around the event and Option of metainfo being Some only when event comes not from replication.
in other words I need not to loose "command context" in event handlers
Martin Grotzke
@magro
Aug 04 2016 06:56
@Tvaroh Are you looking for some kind of context that's available while switching threads/async boundaries?
So this stuff should not be persisted?
Alexander Semenov
@Tvaroh
Aug 04 2016 06:57
no, currently it's a "responder" object associated with a command (that allows to send messages back to the client through websocket connection) and correlation id. I expect to add some more later.
I've introduced a notion of a "transient state" in the eventsourced actor that could only be mutated from command handlers and can be accessed from event handlers if not recovering. So, it doesn't replicate. So, if I store connection id with events, I can take a responder from that transient state in event handlers. But it would not work for correlation id, since I wouldn't know how to clean this information from the transient state (responders are cleaned up automatically in CloseConnection command handler).
Martin Krasser
@krasserm
Aug 04 2016 07:01
The persist handler, which is called after the event has been persisted, anyway closes over its context. It's comparable to use sender() within the persist handler. That should be everything you need
Alexander Semenov
@Tvaroh
Aug 04 2016 07:03
not sure I follow: in event handlers I get raw events, but ideally I would like to pass some non-serializable data from command handlers to event handlers
so, having available a responder in a command handler, I would like to pass it somehow to an emitted event handler (only locally of course)
Martin Krasser
@krasserm
Aug 04 2016 07:04
Yeah, the persist handler is like an event handler that is only called once after successful persistence but not during replay.
From an implementation perspective, it is even called in the same thread
Alexander Semenov
@Tvaroh
Aug 04 2016 07:06

ah, you mean this

      persistN(events) {
        case Success(_) ⇒
          onFinish()
        case Failure(t) ⇒
          logPersistingError(events, t)
      }

?

Martin Krasser
@krasserm
Aug 04 2016 07:06
A common use case for the persist handler is to reply to the command sender when persistence was successful (or failed)
Umm, yes, that's a persist handler example (the partial function I mean)
In the persist handler you can access current state, can access sender() (which is the command sender) or access command-specific data
The persist handler is executed on the actor's dispatcher thread
Alexander Semenov
@Tvaroh
Aug 04 2016 07:17
awesome Martin, thank a lot
Martin Grotzke
@magro
Aug 04 2016 08:09
@krasserm Nice to see you, hopefully you had a good time until now! :-) Do you have an idea regarding the previous question on async event handlers (from Aug 01 07:38 - just don't see how to get the permalink)?
Alexander Semenov
@Tvaroh
Aug 04 2016 08:14
:+1:
This is a similar situation when event handler needs to access responder object. To refresh: external graph database is responsible for validations in my case. I don't use validation in command handlers since I can't make my complex graph an in-memory write model. So I just emit events and handle errors (in event handlers) from graph API. And I need to respond to original command's sender with success or failure objects (in event handlers).
and the graph API is asynchronous
Alexander Semenov
@Tvaroh
Aug 04 2016 08:19
I can get a responder object in an event handler by having a connection id in my event type (I store a Map[ConnectionId, Responder] in my so called transient state). Just want to make sure I'm doing it all right with async stuff
btw, is persist handler called after event handler completes? Or just after persisting an event?
Martin Krasser
@krasserm
Aug 04 2016 08:23
@magro Everything fine here, wonderful tracks :smiley:. I'll take a closer look when I'm back from vacation. Initial thought was that the described patterns are related to the implementation of PersistOnEvent but I only skimmed quickly over the discussion. http://rbmhtechnology.github.io/eventuate/user-guide.html#event-driven-communication and http://rbmhtechnology.github.io/eventuate/latest/api/index.html#com.rbmhtechnology.eventuate.PersistOnEvent give more details
@Tvaroh it's called after the event handler completes
Martin Krasser
@krasserm
Aug 04 2016 08:31
@Tvaroh Are you sure that Eventuate is the right tool for your use case? If you want to make conditional updates to a graph and want to inform the client about the outcome, why don't you use the graph database as your write model and make updates in the DB directly? If you want to track the changes made to your DB, you still can run queries can compute deltas and store them elsewhere.
Alexander Semenov
@Tvaroh
Aug 04 2016 08:38

Martin, I could use the graph database as a write model but I don't want to introduce a period of uncertainty between validation in command handlers (e.g. check if parent exists) and insert in event handlers. In a concurrent situation this might fail and that validation check seems unreasonable since my graph API returns all the information required on insert/etc.

Or you mean without Eventuate?

sorry, not sure if I follow the last sentence though
at least I need replication that Eventuate gives, and it even works :+1:
Martin Krasser
@krasserm
Aug 04 2016 08:43
Yes, I mean without Eventuate. And with the last sentence I mean implementing a CDC interface for your Graph DB (if it doesn't already provide one)
Alexander Semenov
@Tvaroh
Aug 04 2016 08:44
sorry, what is CDC?
Alexander Semenov
@Tvaroh
Aug 04 2016 08:46
how about replication? I have a case when a user on one node changes something in the graph and I need to find users interested in those changes connected to any node and inform them about the change
also with Eventuate I can live without replication at the level of the db (that requires enterprise version in case of Neo4j), and rely on Eventuate to replicate data between nodes
Martin, do you think sending replies from event handlers (when not recovering) is ugly?
Martin Krasser
@krasserm
Aug 04 2016 09:01
@Tvaroh As @magro already mentioned, it's hard to discuss (or help you with) solutions without having a clear context. Regarding replication, do you want a master/master setup or a master/slave? In case of master/master how would you handle conflicting updates? And so on ... If you want us to help you please give us first a concise description of your system model and requirements. If you want to have more extended help regarding system design and implementation, I can also offer professional support (in that case, please contact me directly via email). Thanks!
Alexander Semenov
@Tvaroh
Aug 04 2016 09:02
Got it, I will post a detailed description, sorry for not doing that already
Martin Krasser
@krasserm
Aug 04 2016 09:03
Great, will take a look when I'm back. Cheers!
Alexander Semenov
@Tvaroh
Aug 04 2016 09:03
thank you
Alexander Semenov
@Tvaroh
Aug 04 2016 12:10
Looks like asynchronicity in event handlers breaks persist handlers, am I right?
Alexander Semenov
@Tvaroh
Aug 04 2016 14:19

I'm writing a prototype of a service allowing, in a nutshell, to edit trees of text. The trees are stored in Neo4j as a graph. There is a single tree starting from the root node and each user gets a sub-tree within that tree. Of course he only knows about his sub-tree (though some sharing will be allowing).

Client app (browser for now) uses websocket connection to the server and sends commands to it. Some commands are "create node (relatively to a parent or a sibling)", "change node text", "move node", "delete", etc. Server responds with event-like responses, like "node created", "node updated", etc. A client can set a correlation id on a command and have it on a response to that command. Server uses akka-http to handle websockets.

I'd like to have multiple "app" servers (all masters), and a client can connect to any. Each server has its own backing Neo4j and Cassandra as an event store. Each app server should know what clients are connected to each of other servers, so when a shared node is changed, it should be able to find users on the wire that can see that node and notify them about the change.

Technically I maintain connections information in an EventsourcedActor state, and events like "connection opened", "connection closed" are replicated. The API to the graph is asynchronous. Currently, I don't understand how to protect the data in different Neo4j database from diverging. If I don't have a write model to validate against, then one "move tree" command might come to one server and another "delete node" to another and it might imply a scenario when on one app server data is first moved then deleted (nothing to delete) while on the other - first deleted and them move fails. So I probably should have Neo4j as a write model to validate against, but I don't see how it can help with such conflicts resolution coming from unrelated users.

Another concern is that I can't be sure after validation that another concurrent event handler doesn't change something that would make event persisted by just validated command to fail (when its event handler or some EventsourcedWriter runs). For me it implies that such a validation can't be trusted and I should rely on the responses from the graph API when event handlers call it. Though I guess it's not idiomatic to call asynchronous APIs in event handlers, check their responses and, in my case, reply to the user back about failure or success.

Does it all make sense?

I can't afford more than 1-2 hour of Martin's professional support per week, so any help is appreciated. :)
Alexander Semenov
@Tvaroh
Aug 04 2016 15:49
Something to add: I'm not concerned with the data being overwritten during concurrent edits - it's okay for me. But the data should be equal in each graph db instance, this is important. Also, since I need to modify users on the wire about something happening, I can do it only from event handlers because only events are replicated.
Martin Grotzke
@magro
Aug 04 2016 22:39
Ah, blocked in calls the whole day...
@krasserm great to hear, enjoy the time! :-)
@Tvaroh thanks for the description, will go through it and think about it. As I'm on vacation for the next 3 weeks I'm not sure when I'll have time...
Martin Grotzke
@magro
Aug 04 2016 23:38
@Tvaroh can multiple users edit the same subtree, or is only the "owner" of the subtree allowed to edit its tree and other users can read only?