Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 20:47
    IgorFedchenko commented #4085
  • 20:46
    IgorFedchenko commented #4085
  • 15:09
    IgorFedchenko commented #4085
  • 11:24
    Kenji-Tanaka commented #3887
  • 10:45
    nagytech edited #4086
  • 10:45
    nagytech synchronize #4086
  • 10:43
    nagytech opened #4086
  • 09:00
    Horusiath commented #4077
  • 06:31
    Aaronontheweb commented #4085
  • 06:28
    Aaronontheweb commented #4085
  • 06:24
    Aaronontheweb commented #4085
  • Dec 07 11:49
    IgorFedchenko commented #4085
  • Dec 07 10:31
    IgorFedchenko commented #4085
  • Dec 07 08:36
    IgorFedchenko commented #4085
  • Dec 06 20:57
    IgorFedchenko commented #4085
  • Dec 06 20:32
    IgorFedchenko commented #4085
  • Dec 06 20:01
    IgorFedchenko commented #4085
  • Dec 06 19:55
    IgorFedchenko commented #4085
  • Dec 06 16:22
    Aaronontheweb labeled #3997
  • Dec 06 16:22
    Aaronontheweb closed #3997
Bartosz Sypytkowski
@Horusiath
@verilocation not sure, if you're not overcomplicating things, but in this case you need to place geocode router config on every node, where processor actor may call it
verilocation
@verilocation

Right.. ok that works, thanks... Perhaps I am over complicating things? What would be the recommended way of doing things.

If I create a bunch of ProcessActors to process inbound messages. Would you expect them to create one actor each to work with. Or (in my case at the moment) each ProcessActor defines a pooled cluster of [geocode] to use which, I agree, is more complicated

Bartosz Sypytkowski
@Horusiath
I don't know what are your data relationships (and I'm not a big fan of routers), but maybe it would be better to have processes and geocodes as two different pools with reference to each other
verilocation
@verilocation

That sounds reasonable, how can I do that? Currently within my processor I create the geocode actor pool like so

this.GeocodeActor = ReceiveActor.Context.ActorOf(Props.Create<GeocodeActor>().WithRouter(FromConfig.Instance), "geocode");

Which (thanks to your help) I now know is taken from the node that the Processor is running on, not the test client. Would I be able to define them both in the test client but just let them know about each other?

(P.s. any reason why youre not a fan of routers?)
Bartosz Sypytkowski
@Horusiath

I don't like routers, 'cause they introduce too much "magic". If they were fully verified at compile-time, this wouldn't be much of an issue, but as everything is driven by HOCON config at runtime, it's easy to get hard to debug runtime errors.

Regarding ways to communicate:

  1. You can add some roles to cluster nodes to recognize the nodes, where process and geocode actors lives. Then on the each side, you can subscribe to something called cluster events - these are evens triggered when new node becomes visible to the current actor. With that you can filter out proces/geocode nodes and use actor selection to get references to them.
  2. Second option is to use distributed pub/sub - it's a publish subscribe mechanism that works across cluster nodes. So geocode actors simply subscribe to some topic, while process actors will publish to it.
verilocation
@verilocation
Thank you I shall look into it :)
verilocation
@verilocation
Quick Q... we are a small team (4 developers) but we are starting to run into App.Config issues relating to public IP addresses. Is there any recommended method for dealing with different IPs in different App.Configs for different developers? Because when we check in quite often developers will check in the App.COnfig which will mess about with other developers setups as itll overwrite the IP addresses
Peter Bergman
@peter-bannerflow
Maybe IPs in the config could be avoided? What is the reason for having them there?
verilocation
@verilocation
Currently, on our dev machines, we use our public IP so that we can [a] develop independently and then [b] easily tell our lighthouses to talk to each other to check everything works in a distributed manner
Peter Bergman
@peter-bannerflow
Not sure about the setup but when I develop locally couldn't you connect to a local lighthouse as well? And just connect all nodes on loop back as hostname?
I meant when you develop locally, not I :)
Peter Bergman
@peter-bannerflow
But if you really need the IPs in there to do networked things with other developers then maybe the IP could be injected to the hocon at runtime instead.
Gregorius Soedharmo
@Arkatufus
i thought only lighthouses need predefined ip addresses, and the other nodes in the cluster find each other out through gossip?
verilocation
@verilocation
I find that if I dont specify the public hostname on all my nodes, then even though the lighthouses talk to each other and every node discovers each other, I tend to get messages like (and im paraphrasing) "Tried to send message to [public ip] but its being discard because the node is only listening on [local ip]"
But yes @Arkatufus I only make my seed / lighthouse nodes the ones that my other nodes connect to
Gregorius Soedharmo
@Arkatufus
@verilocation oh, so its a problem with nodes behind router NAT?
verilocation
@verilocation
Honestly not sure, it seems that the lighthouses think the messages arent bound for the right place so discards them
Francis Paulin
@paulinfrancis
I just implemented an extension method on IConfiguration (ASP .NET Core), so that you can have your hocon as json in appsettings.json:
public static class ConfigExtensions
{
    public static Config ExtractAkkaConfig(this IConfiguration section, dynamic configSection = null)
    {
        if (configSection == null)
        {
            configSection = new ExpandoObject();
        }

        var expandoDict = configSection as IDictionary<string, object>;

        var configurationSections = section.GetChildren().ToList();
        foreach (var configurationSection in configurationSections)
        {
            if (configurationSection.Value != null)
            {
                expandoDict.Add(configurationSection.Key, configurationSection.Value);
            }
            else
            {
                var exp = new ExpandoObject();
                expandoDict.Add(configurationSection.Key, exp);
                ExtractAkkaConfig(configurationSection, expandoDict[configurationSection.Key]);
            }
        }

        var configJson = "{ akka: " + JsonConvert.SerializeObject(configSection) + " }";

        return ConfigurationFactory.ParseString(configJson);
    }
}
verilocation
@verilocation
Very nice
Francis Paulin
@paulinfrancis
is there an "extensions" repo I could submit a PR to? I am probably neither the first nor the last person to want to use Akka in Core :)
Ash R
@AshRolls
I'm sorry if this is the wrong place to ask, but I have an actor system and I'm not getting the throughput I would expect? 1 top level actor, with 1000 child actors. Each child actor has another 50 child actors. (51001 actors in total) The top two levels listen for all of their respective child actors to send back a message. All messages sent with tells. To send a message 100 times from top to bottom (and then back) takes around 10 seconds. The code is here.... https://github.com/AshRolls/GalaxyGen/blob/akka_speed_experiment_2/GalaxyGenActorPerformanceTester/DefaultTellCounter/DefaultTellCounter.cs
I've tried playing with dispatcher throughput
Aaron Stannard
@Aaronontheweb
@AshRolls tough to tell what the "throughput" should be without hardware
Ash R
@AshRolls
quad core i5-3570k
John Nicholas
@MrTortoise
whats the recommended goto Receive handler these days?
Aaron Stannard
@Aaronontheweb
but just grokking from where the wind blows here
10 seconds to do that sounds reasonable, and I'll explain right now
the biggest cost to processing a message, usually other than the work the actor actually has to perform when it receives a message, is the context switch on the dispatcher
Ash R
@AshRolls
Yes, that was my assumption
Aaron Stannard
@Aaronontheweb
in your case there isn't that much work since it looks like each child is just sending a basic ACK back
Ash R
@AshRolls
yup this is just the test case. I thought the context switching was taking the time, hence the fiddling with custom-dispatcher throughput
Aaron Stannard
@Aaronontheweb
the dispatcher throughput can't and won't affect the context switching overhead in this scenario
what the dispatcher throughput will do is allow one actor to hang onto the dispatcher for longer before it yields to another
but if each actor only has to process a single message at the lowest level of the hierarchy
Ash R
@AshRolls
so back of the hand 1 mill messages per sec on my hardware is around the correct value?
Aaron Stannard
@Aaronontheweb
the other thing about dispatcher thoughput
it only increases the throughput of a given actor if you adjust that setting exclusively
if everyone's throughput setting goes up, no one's does
(insert exceptional edge cases here where the system isn't that busy, etc)
but in a busy system, that's true
so typically where you see a lot of benefit in increasing the dispatcher throughput is when you use it on a small number of actors who receive a disproportionately large number of messages relative to everyone else
Ash R
@AshRolls
I'll try and optimise the architecture to reduce context switching then
Aaron Stannard
@Aaronontheweb
on our CI system we run on a crappy Windows Azure VM
D2 V2
has two cores
a single actor can do about 4.3-4.5 million messages per second
but that's because its dispatcher is constantly running
Ash R
@AshRolls
I saw the pingpong example mentioned in f#
Aaron Stannard
@Aaronontheweb
yeah, the ping pong example is a better benchmark for looking at context switching costs