These are chat archives for akkadotnet/akka.net

26th
Apr 2017
Yip-Yip
@Yip-Yip
Apr 26 2017 01:21
@afmorris you could try a static factory with the iactorref's .. is a bit nicer than actor selection which is recomened for remote system in practice.. Partial constructer DI would be nice ..my 2 cents
Arjen Smits
@Danthar
Apr 26 2017 08:07
@bdparrish netcore support is currently being worked on
Maxim Cherednik
@maxcherednik
Apr 26 2017 08:29
hi guys, got a q about Akka.Persistence and read model(projection) building. I used NEventStore before. There was a global sequence number. So when I need to create some projection I was subscribing to all events basically with single sequence id to track the state of my projection. As far as I understood there is no such concept in Akka.Persistence and maybe for the best-streams are independent and those are our portioning point. Having all of this, I was wondering how to projection building done? I can try to imagine it this way:
On start of the Projection Service, I read(subscribe) all the stream ids(is there such API?) and for every single one I create a subscription per stream. So as a result I have as many individual projections as we have streams.
Question: is this how it should be done? Something to worry about?
Maxim Cherednik
@maxcherednik
Apr 26 2017 09:15
And one more: what about concurrency check on the Akka.Persistence? Haven't found anything about it.
Arjen Smits
@Danthar
Apr 26 2017 09:41
@maxcherednik what do you mean with concurrency check on Akka.Persistence ?
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:03
2 copies of the aggregate writing to the same stream
on NeventStore and GeteventStore there is also a version column so that to tell the caller that Aggregate is outdated
Arjen Smits
@Danthar
Apr 26 2017 11:12
Well.. how would you handle that in any other EventSourced system? That uses Aggregates ?
hint: You don't do that :P
How would your aggregate be able to enforce its consistency rules
when you have 2 sources of truth ?
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:14
hm... maybe I described it not really clear :)
Arjen Smits
@Danthar
Apr 26 2017 11:14
In NEventStore or the likes, this is enforced by the version column
however those libraries throw an exception when you have concurrent access to the same stream
as in, when it cannot merge the 2 different commits
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:14
yest
Arjen Smits
@Danthar
Apr 26 2017 11:14
in Akka
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:14
this is what I need :)
Arjen Smits
@Danthar
Apr 26 2017 11:14
your actors are statefull
so it does not need this kind of check on the persistence level
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:15
good :) but by mistake I created 2 or more copies of the same actor
Arjen Smits
@Danthar
Apr 26 2017 11:15
because by design, each persistent actor has his own unique persistence iD
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:15
or course
Arjen Smits
@Danthar
Apr 26 2017 11:15
having multiple actors instances running with the same persistence id, is not supported
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:16
we are on the same page here
so that's what I want to know
if by any means someone creates several copies of the same Actor with the same ID
the state is broken and there is no way out
Arjen Smits
@Danthar
Apr 26 2017 11:16
Then you have a problem.
Im not sure if there is a check for that in a local actor system
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:17
I might guess you might propose this Singleton of the cluster
Arjen Smits
@Danthar
Apr 26 2017 11:17
@Horusiath would know for sure. He knows akka persistence inside out
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:18
but it's not good enough I guess
Arjen Smits
@Danthar
Apr 26 2017 11:18
Well actors always live in a hierarchy
So usually you have some kind of actor sitting in front of your persistent actors
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:19
Yeah but in the distributed system there is always a possibility to have several copies
Arjen Smits
@Danthar
Apr 26 2017 11:19
which can be used to get a reference to a specific persistent actor, or act as a gatekeeper for messages going to that pool of persistent actors
Well in a akka cluster there are various ways of managing that
cluster sharding is one of them. Distributed hash-group routers is a (simpler) second one.
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:22
hashed router and and sharding are cool, even though not removing the possibility, right?
Arjen Smits
@Danthar
Apr 26 2017 11:23
Well. In those cases, when would a 2th (duplicate) actor be started?
When the hash-group suddenly starts routing messages to a new node, because the existing node is down
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:24
yeah, as a result of split brain or something like this
Arjen Smits
@Danthar
Apr 26 2017 11:24
well split brain is a specialist case, so lets shelve that for now
the existing node is down. Due to network issues or simply that the machine died
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:25
maybe died:)
Arjen Smits
@Danthar
Apr 26 2017 11:25
In both cases, you are going to restart that node instance.
but meanwhile due to the hashing router, messages are routed elsewhere
a new actor has been started
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:25
ok
Arjen Smits
@Danthar
Apr 26 2017 11:25
its state has been rehydrated, and live goes on.
the original actor, is long gone
So even if there are 2 duplicate actors alive
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:26
ok, this is so called happy path
Arjen Smits
@Danthar
Apr 26 2017 11:26
i would postulate, that if you have your design right. One of those actors are actually always on node that is marked down
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:26
if everything like we plan
Arjen Smits
@Danthar
Apr 26 2017 11:28
Well then what is the unhappy path, in your mind ?
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:28
I don't know :) I don't really have much of an experience with Akka cluster. I just have this feeling :)
Arjen Smits
@Danthar
Apr 26 2017 11:29
ok.
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:29
just looking for a proof from the community
Arjen Smits
@Danthar
Apr 26 2017 11:29
haha, well thats going to be hard.
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:29
like: if you do it so, this would be nearly impossible
lets say split brain is one of those case
Arjen Smits
@Danthar
Apr 26 2017 11:30
Akka.Cluster offers all kinds of mechanisms to handle these cases, through routers, akka.clustersharding, AtLeastOnceDelivery actors, etc
But short of bugs, or a fault in your design. Im not familiar with any glaring problems on that end in the akka ecosystem.
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:31
btw on the split brain will hashed router rebalance?
Arjen Smits
@Danthar
Apr 26 2017 11:31
Yes
With a split brain you potentially have 2 clusters. Which is an issue.
However the hashed router lives on 1 node
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:32
so there will be 2 clusters then - and 2 commands might reach 2 different versions of the aggregate
Arjen Smits
@Danthar
Apr 26 2017 11:33
so whichever node the hashed router lives on, will be the cluster where your work is rebalanced
the other cluster, will simply be unable to deliver the message
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:33
how so? I thought that hashed router lives on every node you need to be? I even so it on the documentation
Arjen Smits
@Danthar
Apr 26 2017 11:33
Ah yes your right. It will be recreated
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:33
let's say I have 3 web nodes
and 3 worker nodes which are behind the hashed router
on every web node I have a copy of router
Arjen Smits
@Danthar
Apr 26 2017 11:35
But split brains are never random
as in, the 'fault-line' that runs across your cluster. is never random
its always pretty well defined
if your using hashed routers to distribute work. What you would normally do is run those on nodes with a specific role.
and have those distribute work on nodes with a different role
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:37
yeah.. something like this
Arjen Smits
@Danthar
Apr 26 2017 11:37
What i would do in such a case, if split brain is a real threat (as in, your not running all nodes in a local network on the same hardware site)
lets say you have 3 management nodes (where the hashed router lives) and 6 worker nodes
you could partition your nodes in such a way, that all management nodes always live in the same data center
so if something happens, there is a good chance those nodes will still be connected.
But this is all speculation. Because there are alot of other ways split brain can occur.
Bottomline is, Split brain resolving always needs human intervention
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:39
to be honest, I've never ever experienced a network partition, but I hear lots of stories when it might happen even within one data center
Arjen Smits
@Danthar
Apr 26 2017 11:40
And the way you should design your system to prevent problems in highly statefull clusters, is always tricky. And can differ wildly depending on what kind of work your doing, or how your network topology is structured.
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:40
I don't know the likelihood of this
I just like the model of the geteventstore and neventstore
cause it's easy to comprehend
and somehow gives you guaranties
I wonder why didn't they implement it in the first place - it seems easy to implement within one stream
could it be that they started akka persistence not having CQRS in there heads ?
Arjen Smits
@Danthar
Apr 26 2017 11:44
No idea.
Maxim Cherednik
@maxcherednik
Apr 26 2017 11:48
ok. thanks for the input. anything on the first question?
Bartosz Sypytkowski
@Horusiath
Apr 26 2017 16:19
@maxcherednik
  1. In akka persistence there's no global sequence number - each stream (identified by persistenceId) has its local sequence nr. This is mostly because global sequence number is not viable option in distributed systems without crippling performance. For the same reason, currently there's no API inside read journal to get live stream of all incoming events: however there's such proposal for databases, that don't support sharding.
  2. Also you shouldn't write to a single event stream concurrently from multiple actors. This will corrupt event stream. In akka this isn't usually a problem, as you may define a simple actor working as aggregate and provide all event stream data through it - if you need simple distribution in case when you have a lot of such actors, you could i.e. use cluster sharding.
Maxim Cherednik
@maxcherednik
Apr 26 2017 19:36
@Horusiath thanks a lot. Re: projections. So the right approach is to listen for the stream of the stream ids and for each stream id I should create actor, which subscribes for the corresponding stream? What if there are too many?
Bartosz Sypytkowski
@Horusiath
Apr 26 2017 19:38
@maxcherednik right now this is very limited and inefficient approach. If you need to just get a subset of all events across different streams, you could wrap your events using Tagged event with tags attached to it. The you can make projections based on specific tag
Maxim Cherednik
@maxcherednik
Apr 26 2017 19:41
Hm... I thought that would be more or less ok. Cause I don't see the other way of doing the projection here. I can't think of a specific tag. All I know, I have streams and I need to have the latest state in memory.
In fact, I am just guided by the API of the specific eventstore. For instance, NeventStore and geteventstore they don't have an API, which returns list of streams, but they have a global sequence.
Bartosz Sypytkowski
@Horusiath
Apr 26 2017 19:43
but state of what? Global world state?
Maxim Cherednik
@maxcherednik
Apr 26 2017 19:44
Difficult to say which state. I am trying to pick the right way. So far state of a stream seems easier!
Bartosz Sypytkowski
@Horusiath
Apr 26 2017 19:46
I mean, state should serve some goal. Iterating over stream of all events just for sake of iteration doesn't serve any purpose
Maxim Cherednik
@maxcherednik
Apr 26 2017 19:46
And also easier to scale if there are many. The only concern of mine is starting time. How efficient the GetAllStreamIds API?
Of course!
Bartosz Sypytkowski
@Horusiath
Apr 26 2017 19:49
btw. which persistent backend are you using?
if you have sql or Eventstore, you could probably run projections by yourself
Maxim Cherednik
@maxcherednik
Apr 26 2017 19:50
After all I need to have all the streams in memory(there is a cleaning logic of course), so that I could do some real time calculations. And for me is not really important if it's a global state or set of stream states. Second one seams easier, since streams are "living" independently
So far I am using Neventstore and global sequence. Which I don't really like. Looking into akka. So considering options.
RichardLindeberg
@RichardLindeberg
Apr 26 2017 19:59
What do you mean with having the streams in memory? Do you want to do arbitrary calculations on the stream when you need the value?
Maxim Cherednik
@maxcherednik
Apr 26 2017 20:00
I mean that I need some projection of a single stream.
And I need all of projections like this.
So that I could do the sum of those results.
Or other kind of calculations which including some of those projections.
Maxim Cherednik
@maxcherednik
Apr 26 2017 20:05
Basically you can think of this as an in-memory read-side. Why in memory? Cause I need to react on every new event on every stream. One of the streams changed? I need to recalculate it's value, so the value of the sum of those projections.