These are chat archives for RBMHTechnology/eventuate
Global-scale event sourcing and event collaboration with causal consistency
!recoveringin the event handler for at-most-once semantics or use
ConfirmedDeliveryto let another actor execute the side effect with at-least-once semantics
Thanks, @mehrentreich My idea is that if actors are indeed cheap (which they seem to be in terms of cpu / thread usage), and i have a read model (Eventsourcedview) that uses a "big" chunk of memory, i could probably obtain better/cheaper scalability if i can have more cpu capacity working on the same shared in-memory (immutable) data structure, computing several queries concurrently. The relocation of the data from memory to a separate database (or elastic search as you suggest) could be a way to do that since that would then be the "only" place containing the shared query state (although in that setup the query results will still need additional memory on each actor). But it seems a bit of an overkill for me because i expect the data that needs to be used for these queries to be a fairly constant 100-200mb in size and that could easily fit on memory leading to faster and cheaper queries with less latency.
The problem is that if i need to fully clone the Eventsourcedview actor(s) then the scaling of cpu and the scaling of the memory will be tightly coupled and i would need a replica of that 100-200mb for scaling cpu by adding additional actors. So even if actors are lighter than threads, the memory usage with that approach will still lead to bad scalability. From my research i found some places saying that sharing mutable actor state is a bad practice (http://manuel.bernhardt.io/2016/08/02/akka-anti-patterns-shared-mutable-state/) and there are also issues with the actor location (since shared state will not work across locations). But if the shared state is not mutable (or delays on the mutations to propagate to all queries are not important) it does not seem like a bad idea to me to have additional copies of the state only on different locations and have several actors sharing state at the same location / node.
Maybe i try to optimize too much :D, but i have been googling about this on the past few hours and it seems like Akka Agents (http://doc.akka.io/docs/akka/current/scala/agents.html) and ScalaSTM (http://nbronson.github.io/scala-stm/) may be a way to do this.
For now i think i will just start trying some implementations until i find one that goes well enough...
EventsourcedView) could maintain the read model in an immutable data structure and return a reference to it on request so that the actual query execution can be done on this data structure outside of the actor and thus concurrently. Keep in mind that snapshots might be required for such a view to keep replay times short when the event log grows. The alternative of a persistent view (as proposed by @mehrentreich ) maintained through an
EventsourcedWriterwould not require snapshots.
Thanks @volkerstampa! that actually looks like a very good design. Just to clarify what you explain: if an actor replies a message with a reference to the query structure, only the reference is passed around with no serialization / deserialization involved and no additional copies being made of the structure? I haven't learned how Akka implements the message passing internally, but if it is optimized to only pass a reference to the same memory (when possible due to actor location) that could actually be perfect for my use case.
For the snapshots vs EventsourcedWriter i have not a sufficiently clear idea yet of what i need but i will certainly study both options. It may even be enough to just use replayFromSequenceNr for some of the views i have (in particular the view that needs to have info about all the aggregates for querying) because i think i do not need events older than some "close" date for that specific query view and it may be possible to know / store which sequence number corresponds to that date and replay only from that point forward. So i guess i will probably need at the end a combination of snapshots, sequence numbers and (maybe) persistent views for different parts of the project or different microservices.