These are chat archives for akkadotnet/akka.net

12th
Feb 2017
TonyLo1
@TonyLo1
Feb 12 2017 11:41
Performance issue: An Akka system running on desktop with 16 (logical) cores runs 4x faster than same system on 88 (logical) core server (split into 2x NUMA nodes of 44 cores). The application generates in excess of a million actors so there is plenty of work to go around. There is no I/O involved, all memory based system. Server is configured in app.config to use all cpu groups and gcserver. Definitely using both NUMA nodes on Server. Is there anything that can be configured to improve this? Any ideas? thx
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 12:00
@TonyLo1 hard to say, I suppose it's more related to your actor topology. I remember someone tested a simple actor topology on 8 vs 48 cores. Results where somewhere close to 21-25 (on 8 cores) vs 127 (on 48 cores) mln messages passed per sec. So while this is not exactly linear progress, it was pretty close to linear.
Olivier Mühring
@fysicus
Feb 12 2017 12:04
A more general question, not certain this is the proper place to ask such questions. :-) So far I've worked with Akka .NET and with Service Fabric. I'm just wondering, does Akka offer a framework which compares to the functionality Service Fabric offers out of the box?
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 12:07
@fysicus depending on which functionalities you're talking about. Service Fabric is not even a framework, it's a whole runtime.
Olivier Mühring
@fysicus
Feb 12 2017 12:11
@Horusiath Yes, I know. So I expect Akka would offer the package as a framework. Just wondering if there is such a thing. Don't have a favourite myself. Service Fabric is nice and all... but it seems a bit over the top for smaller applications.
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 12:11
you can have automatic actor lifecycle management like SF offers with cluster sharding. I've seen people, who integrated akka.net cluster with SF service discovery mechanism. There are no reliable collections, but there are CRDT data types, which offers a bit more features. Persistence between SF actors and Akka actors is different - akka uses eventsourcing by default, AFAIK service fabric can only support snapshotting. And AFAIK in terms of performance Akka is way faster (at least in local communication) than SF
but it all depends on your goals and objectives
Olivier Mühring
@fysicus
Feb 12 2017 12:14
@Horusiath True :) I was just wondering. :-D
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 12:14
people who start their story with distributed, clustered systems, will probably catch and start developing with SF faster.
(ofc after installing the whole SDK)
Olivier Mühring
@fysicus
Feb 12 2017 12:20
Well, last year I worked on small scale service, akka seemed a perfect fit for those. Though Azure does make certain aspects (like remoting) difficult.
Currently I'm working on a multi-tier solution, where my teams part consists of processing all data related requests.
Running on Azure, with service app fabric
Here peformance and reliability are extremely important, probably why Service Fabric was selected since this does a lot of the background work by itself.
TonyLo1
@TonyLo1
Feb 12 2017 12:35
@Horusiath , re: performance issue, I think you are right, Just ran the pingpong benchmark and it is close to linear speedup (allowing for clock speed variance). Getting approx. 180 mln msgs/sec so it is working fine. Need to look at my architecture. Thx
Maxim Cherednik
@maxcherednik
Feb 12 2017 18:07
Hi guys, reading a bit into akka streams and still can't quite get it. The overall idea is quite clear, but all the examples are rather simple and single block only. So I am trying to understand if this framework could a fit for my task. Let me describe a bit: let's say I have an eternal stream of trades. What I need to do is kind a different grouping, aggregating and so on, so the chain could be quite complicated. As far as I can see the stream interface supports only one type - in my case Trade. What if the semantics of my stream is add, update, delete? How would I organise the whole chain? And imagine on top of this lets say I need to group by the instrument and count trades per each group - which will push add, update, delete again... In fact, I was once working in a bank where they had an in-house framework similar to what I described - no back pressure though. So maybe akka streams is not the right tool for my task?
Maciek Misztal
@mmisztal1980
Feb 12 2017 18:18
is there a roadmap for 2017 yet?
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 18:56
@maxcherednik if you need persistence and distribution, then Kafka as an intermediate log would be a good fit. Akka streams can be generally speaking used to any kind of workflows i.e. I've created web crawler workflow graph totally on top of streams. There's more than enough tools to make any kind of operation you've described there, but again, if you need persistence, you'll probably need some event log in front of it.
Maxim Cherednik
@maxcherednik
Feb 12 2017 18:57
I have it. My concern is add/update/delete semantics all over
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 18:58
add/update/delete sounds more like eventsourcing - something better for akka.persistence aggregates (you can use akka.streams and akka.persistence.queries from there to generate readonly views)
but even without it, a state of application is basically a fold of incoming events (or in case of Akka.Streams either Aggregate or Scan ;) )
Maxim Cherednik
@maxcherednik
Feb 12 2017 18:59
so what you are telling is that I'd better go for a usual actors ?
but the thing is I still need a chain of those - filtering, grouping, etc...
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 19:00
actors are more elastic, can be used in more broad cases, akka.streams is designed specifically for workflows working in single actor system
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:00
if I do all of this manually, I will get tired :)
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 19:01
you can combine streams with actors
actor can be publisher/subscriber to the stream
(however making backpressure from there is more tricky for persistent actors)
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:01
sure, but this is not a problem - in and outs are easy :)
so in your opinion my task is not exactly for akka streams? or it's still doable but not straightforward ?
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 19:03
you can use streams, but their limitations may kick you later ;)
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:05
for instance from that in-house framework I remember Join link. It was pretty nice - stream of trades versus stream of ticks - join by intrument - out trade with extra column - price.
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 19:05
@mmisztal1980 simply speaking, we're trying to reach .net core portability (AFAIK we are few PRs away from that).
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:05
then trade removed - and this join immediately affected - it sends remove to the out...
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 19:07
@maxcherednik in akka.streams joins are called .Via methods and operate on stages - stages are basic building blocks of akka.streams, and you can basically do whatever you want from them
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:07
via? I thought it's just to link it to the next block
I read all the docs
Bartosz Sypytkowski
@Horusiath
Feb 12 2017 19:09
it's due to difference in standard stage vs source (initial producer or events) vs sink (ultimate consumer). Via is used on stages, that transform data moving through them, sinks are ultimate consumers and can be joined using .To method.
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:09
ok
so still it seems to me that streams they are more like very simple - one way flow
this is funny, I had very same question some time ago about Rx Extensions
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:16
IObservable<T>
so if I really want to have add/update/delete then I need to have a T which inside contains info about what exactly I am doing - a/u/d
and I don't believe it's an easy thing to implement all of this
Maxim Cherednik
@maxcherednik
Feb 12 2017 19:22
or maybe I shouldn't introduce this update flag and instead I should just push the same Trade but with the updated fields and the consumer is supposed to understand that this is already a newer version of the previously received trade
Maciek Misztal
@mmisztal1980
Feb 12 2017 21:04
@Horusiath awesome, looking forward to it, as I'm now diving into the world of dcos/mesos :D
babafemi
@webafriq_twitter
Feb 12 2017 23:37
Hi all. Just a quick one. If you edit inmemory data in akka.persissstence, and you persist into a journal what happens to the original state of the data, or does it add it as a duplicate? Is this piece of code correct or how is update done?
Command<EditSuperAgentMessage>(str => Persist(str, s =>
{
var superagents = _superAgentMessageStore.Where(x => x.Id == str.Id).Select(x => x).FirstOrDefault();
superagents = new CreateSuperAgentMessage(xxx, xxx);
_superAgentMessageStore.Add(superagents); //add msg to in-memory event store after persisting
SaveSnapshot(_superAgentMessageStore);
}));
Working with is inmemory list:
List<CreateSuperAgentMessage> _superAgentMessageStore = new List<CreateSuperAgentMessage>();