These are chat archives for akkadotnet/akka.net

17th
Mar 2018
TonyLo1
@TonyLo1
Mar 17 2018 12:30
Using Akka.streams, can't get app to run across NUMA nodes in multi CPU servers. The app maxs out the CPU on one node but will not use both NUMA nodes. Is this a limitation of akka.streams or is there some way to change this behaviour? Any thoughts? thnx
P.S. have set the app config to EnableALLCpuGroups, etc
Marc Piechura
@marcpiechura
Mar 17 2018 13:12
@TonyLo1 I don’t know anything about NUMA but Akka Streams has no limitation on how many cores it can run since it’s using actors under the hood.
Onur Gumus
@OnurGumus
Mar 17 2018 13:13
@TonyLo1 I think akka streams use regular .net thread and task pool. The motto of akka is "no magic" :)
Marc Piechura
@marcpiechura
Mar 17 2018 13:13
One issue could be, if you’ve only synchronous stages, i.e. Select, Where ... , that these all run in the same actor and therefore only on one cpu
Onur Gumus
@OnurGumus
Mar 17 2018 13:14
so if I were you I would check if .NET threadpool or TaskPool runs on Numa first
TonyLo1
@TonyLo1
Mar 17 2018 13:20
@marcpiechura, @OnurGumus thanks for the response. The app is not only using synchronous stages as the app is happy using 44 cores on one CPU - it just wont use any of the cores on the other CPU, i.e, the other NUMA Node - one complication is that once the number of logical cores is greater than 32 per CPU the OS creates CPU groups. To ensure all CPU groups are used you need to add EnableAllCpuGroups to the app.config, which I have done. So I'm wondering if this is an issue with akka using multiple CPU groups as this is an unusual scenario.
I have other .net apps which use both NUMA nodes so the threadpool does support it
Bartosz Sypytkowski
@Horusiath
Mar 17 2018 13:25
@TonyLo1 it's strange, I remember people using akka.net on 128 core machines - I guess they were probably usng NUMA too - with full core utilization.
TonyLo1
@TonyLo1
Mar 17 2018 13:39
@Horusiath, yes it's strange. It may be my NUMA setup but I have had both CPUs running with other .NET apps so I'm not convinced this is the issue.
I misspoke re: the app.config setting - it should be Thread_UseAllCpuGroups = "true" which I have, so this is not the issue either
TonyLo1
@TonyLo1
Mar 17 2018 14:00
Ok, problem solved. Increasing the number of top level streams uses both CPUs. Interestingly increasing the number of low level streams just caused one CPU to max out. So it is architecture sensitive. Thanks for all the input.