These are chat archives for akkadotnet/akka.net

4th
Apr 2016
Kris Schepers
@schepersk
Apr 04 2016 09:22
@Horusiath Playing with Sharding, I just encountered the following: 2 nodes up, both hosting a region, both processing shard messages. One node leaves cluster, after a few seconds I restart it, it joins the cluster again, but it doesn't receive any shards. The node that didn't shut down keeps on processing all the messages for the entities. So it seems the rebalancing of the shards to the rebooted region doesn't happen?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 09:24
@schepersk rebalancing check is triggered after X seconds, and some specific threshold needs to be reached in order to start rebalancing
in case when the difference between number of shards on two machines is too small
Kris Schepers
@schepersk
Apr 04 2016 09:25
10 shards, 2 nodes in my case?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 09:26
also: are you sure, that your cluster node really rejoined?
Kris Schepers
@schepersk
Apr 04 2016 09:27
from what I can see in the logging, yes.
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 09:28
can you extract some example to reproduce from that?
This is the sample I'm working on..
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 10:28
@schepersk I'll managed to run your example, but to verify it, I'll need to find some spare time. Sorry, but it may take a while to find a reason for this. I'll take a look into it after work
Kris Schepers
@schepersk
Apr 04 2016 10:35
@Horusiath But you are seeing the same behavior? Thanks for looking into this..
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:31
How lightweight are agents?
Zetanova
@Zetanova
Apr 04 2016 13:37
Agents or actors?
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:38
actors sorry
I was thinking if they are lightweight enough to use them for agents in a multi agent system.
I would assume though that if you would like to do that you would write your own dispatcher for it.
Zetanova
@Zetanova
Apr 04 2016 13:39
not big, i little bigger then a normal object, plus some registration in lists
yes, the are very light in this sence
over a count of 100k you would need to think about memory
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:40
Ok :)
Zetanova
@Zetanova
Apr 04 2016 13:41
but i dont know the exact byte count for each in avg.
i am loading ~1000 per each aggregate root type
and the app has still low memory consumation
messages are short lived and most likly removed on Gen0 GC
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:45
Are there any multi agent systems that use Akka.net?
Zetanova
@Zetanova
Apr 04 2016 13:45
multi tenant?
Zetanova
@Zetanova
Apr 04 2016 13:46
yes, akka.net is perfect for it
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:47
But you have shared state in form of the world state.
Zetanova
@Zetanova
Apr 04 2016 13:47
or an other actor-model system
world => actor
can be a singleton service t0o, but better an actor
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:47
Ok but i guess you need to throw away determinism then.
Zetanova
@Zetanova
Apr 04 2016 13:48
not realy
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:48
Lets say you wanted to model something like a board game.
Then you need the sequence of actions of several of the actors.
Zetanova
@Zetanova
Apr 04 2016 13:48
then the board can sync the steps or a child of board-actor
Nicolas Bourbaki
@b0urb4k1
Apr 04 2016 13:49
Oh!
What is the name of the class i need to look up?
Zetanova
@Zetanova
Apr 04 2016 13:51
?
ActorBase
most likly
Christian Duhard
@cduhard
Apr 04 2016 16:17
If I send a RoundRobin router a PoisonPill.Instance i assume they all get the PoisonPill?
Christian Duhard
@cduhard
Apr 04 2016 16:24
are there any examples on how to use GracefulStop correctly?
Aaron Stannard
@Aaronontheweb
Apr 04 2016 16:37
@cduhard if you PoisonPill the router the router dies
and so do all of its routees
if it's a pool router
Christian Duhard
@cduhard
Apr 04 2016 16:37
k it is
Aaron Stannard
@Aaronontheweb
Apr 04 2016 16:37
for using GracefulStop
how that works by default is it PoisonPill's the target actor
Christian Duhard
@cduhard
Apr 04 2016 16:38
@Aaronontheweb i am mostly not sure what to do with the Tasks if I have numerous children to stop
Aaron Stannard
@Aaronontheweb
Apr 04 2016 16:38
and you the Task<bool> returns true if the temporary actor created by the GracefulStop call, kind of like the temporary actors created by Ask, gets a Terminated message
Christian Duhard
@cduhard
Apr 04 2016 16:40
say for instance I have
Actor2.GracefulStop(TimeSpan.FromSeconds(10));
Actor3.GracefulStop(TimeSpan.FromSeconds(10), new Shutdown());
when all gracefulstop Tasks are complete I need to send a Passivate type message
i guess this is more about my shitty async skills than anythig
Christian Duhard
@cduhard
Apr 04 2016 16:45
I assume sending a poisonPill just effects that actor and not it's watched children
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:20
@alexvaluyskiy that's the goal
Christian Duhard
@cduhard
Apr 04 2016 17:38
@Aaronontheweb so the bool returned from GracefulStop just tells me if the stop was actually graceful and didn't time out?
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:38
bool == false if it timed out
didn't shut down in time
bool == true if it did
Christian Duhard
@cduhard
Apr 04 2016 17:39
so its still running if it's false?
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:39
and after we merge akkadotnet/akka.net#1765 all children will have been terminated also
like, guaranteed to have been stopped
correct
still running if false
when PoisonPill is sent, the actor must process all previous messages in its inbox first
before it can shut down
you can customize the shutdown message on GracefulStop too - there's an overload that supports that
so it can be something user-defined potentially
Christian Duhard
@cduhard
Apr 04 2016 17:41
yeah, i've been using that
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:41
just know that your message will also be user-defined, so it'll have the same behavior as PoisonPill in terms of processing order
if there's a big backlog of messages sitting in front of it
Christian Duhard
@cduhard
Apr 04 2016 17:42
yep
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:42
then the likelihood of a timeout will be higher
Christian Duhard
@cduhard
Apr 04 2016 17:42
this graceful shutdown is a lot harder than I anticipated
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:42
what's the issue exactly?
Christian Duhard
@cduhard
Apr 04 2016 17:47
coordinating everything
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:48
one pattern I've used
and we use this inside the Remoting class for doing a simultaneous shutdown of all transports at once
is I'll build a list of IActorRefs that need to be shutdown
call graceful stop on all of them and build a Task<bool> []
Christian Duhard
@cduhard
Apr 04 2016 17:48
my persistent actors need to passivate before I can shutdown a bunch of supporting actors
Aaron Stannard
@Aaronontheweb
Apr 04 2016 17:49
then I'll call Task.WhenAll and log a warning if not all of those actors shutdown
ah, got it
yeah, it's tricky
Christian Duhard
@cduhard
Apr 04 2016 18:07
i also have a tricky saga type thing in my persistent actors
Christian Duhard
@cduhard
Apr 04 2016 18:17
                    var m = message as Shutdown;
                    while (Context.GetChildren().Any(x => x.GetType() == typeof (Account)))
                    {
                        Thread.Sleep(500);
                    }
that seems pretty ugly
Account is the persistent actor type
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:17
@cduhard belive me, you never want thread blocking in your actors logic. Like NEVER
Christian Duhard
@cduhard
Apr 04 2016 18:18
i was more demoing the line of thinking. i have never ran that code
@Horusiath baby kittens are safe
Pavel Knorr
@knorrus
Apr 04 2016 18:19
Hi guys! Observing strange issue with Akka persitence, my configuration looks like:
Akka.Persistence 1.0.6.17-beta
Akka.Persistence.PostgreSql 1.0.5.2-beta
In code I have following:
https://gist.github.com/knorrus/c2575fd893f6432d2ee15e2b1750b3b2
Shortly when sending something to journal everything is ok, but during recover, Inner field of de-serialized object is always null
Is there is a know issue?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:21
@knorrus this is problem of:
  1. Json.NET v7 being unable to correctly serialize/deserialize auto properties initialized with C# 6 syntax
  2. Using JSON.NET at all as the default serializer for persistence ;)
Christian Duhard
@cduhard
Apr 04 2016 18:21
are there other good options for json?
Pavel Knorr
@knorrus
Apr 04 2016 18:22
@Horusiath I'm using Newtonsoft.Json 8.0.2
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:22
json.net v8 has fixed this issue, but you shouldn't relly on the default akka serializer for persistence - it was not created for that
@knorrus try public UserFile UserFile { get; private set; } to verify
Christian Duhard
@cduhard
Apr 04 2016 18:23
@Horusiath what would you recommend to accomplish what my terrible thread.sleep example does?
Pavel Knorr
@knorrus
Apr 04 2016 18:24
@Horusiath YOU SAVE MY DAY!
with private setter it works
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:28
@cduhard first you'd need a good way to recognize children of type Account - potentially you could use actor name for that. Then either send the account shutdown message to all children (and make only account able to handle it) or use selection to filter account children and send shutdown to them
Pavel Knorr
@knorrus
Apr 04 2016 18:28
@Horusiath Which serializer should I use for persistence? Akka.Persistence comes with Json.Net by default
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:28
with account children recognized by name, you could easily count them and wait until all of them are dead
Christian Duhard
@cduhard
Apr 04 2016 18:29
@Horusiath They already passivate on their own
@Horusiath I just need a way to wait until they are all terminated before starting shutdown of supporting actors and routers
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:31
@knorrus the major problem with the default serializer is that it may (and will be) changed in the future, while your persisted events format won't. You can always use your own serializer - depending on who you'll ask, there will be different responses. My favourite are schema-defined ones, like proto buf, flat buffers or MS Bond
@cduhard it's still better to do so asynchronously
Christian Duhard
@cduhard
Apr 04 2016 18:33
@Horusiath i understand. How to do it async is the part I am having some trouble with. I was hoping to use built in mechanism rather that add a bunch of new message flow
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:35
@cduhard so you want to stop a parent once all of it's children of specific type will answer to stop request?
to11mtm
@to11mtm
Apr 04 2016 18:35
@Horusiath : Wasn't the expectation that things would be moving to 'Wire'? Or has that changed?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:35
@to11mtm yes, that's the plan
Christian Duhard
@cduhard
Apr 04 2016 18:36
@Horusiath The AggregateCoordinator maintains many worker routers that are used by th persistent actors. They can't be stopped until all are passivated
to11mtm
@to11mtm
Apr 04 2016 18:36
Would that be a fairly safe way to go as far as if our team was planning on setting up persistence now?
Christian Duhard
@cduhard
Apr 04 2016 18:39
@Horusiath i need to order the shutdown -> PersistentActors -> All Other Children (can be in parallel)
i suppose I could start a timer when the Coordinator receives the Shutdown which is what the silly thread.sleep us trying to accomplish
Become(ShuttingDown) -> fire up timer, handle the timer message -> check for PersistentActors status, then shutdown remaining worker children
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:42
@to11mtm in general you should never allow third party library tell you what format it will use to store your data. What if something changes or you'll need to migrate your data to another solution? You should either specify message format by yourself or have some migration path for it - if I'm right after 1.5 we'll switch to Wire, but still you'll be able to use json.net serializer as an alternative, you'll just need to specify it explicitly. Also we'll probably provide some tips about how to write format migration scenarios.
Christian Duhard
@cduhard
Apr 04 2016 18:42
i think that will work. see any flaws?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:43
@cduhard what do you need timer for?
Christian Duhard
@cduhard
Apr 04 2016 18:43
to check for non-terminated persistent actors
i can't complete the shutdown until they are all passivated
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:44
this is something, you should be able to use poisonpill + terminated watch guard for
Christian Duhard
@cduhard
Apr 04 2016 18:44
i don't see how
an external shutdown gets triggered, Coordinator receives and waits for passivation of all PA children. once they are all terminated shutdown of supporting worker children can commence
its the waiting part that requires the timer
i can't use a poisonpill because I can't shutdown a PA that way
there are non-persistent sagas in progress
so i can just wait for all to passivate
if I had your cool persistent sagas in your new cluster sharding sample app, I could just poisonpill the lot and move on
Christian Duhard
@cduhard
Apr 04 2016 18:50
the current passivation timer waits for those sagas to play out, e.g skips passivation if there are any in progress
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:51
then you can create your custom PoisonPill equivalent, that can be rejected or delayed due to saga being in progress
Christian Duhard
@cduhard
Apr 04 2016 18:52
but then I still have to wait
I don't need to build that because I can just wait for them to passivate. there are no new messages coming in. the shutdown has cut that off further up the hierarchy
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:53
you can wait for termination
Christian Duhard
@cduhard
Apr 04 2016 18:53
yes, I need to know when all are terminated
so i can move forward
a timer is just as easy as trying to keep track of what's alive etc
not as precise, but far less complex
if i were to cluster & shard this I would do this in a more robust way
am I making sense?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:55
why you thing that handling Terminated is more complex?
Christian Duhard
@cduhard
Apr 04 2016 18:55
there is a disconnect here
i need to know when all of them are terminated
i can't do anything until that happems
*happens
they terminate without the app being in shutdown
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 18:56
...
  // normal receive
  message.Match()
  .With<Shutdown>(shutdown =>{
    var accounts = Context.GetChildren().Select(aref => aref.Path.Name.Contains("account")).ToArray();
    foreach(var account in accounts) account.Tell(shutdown);
    Become(WaitingForTermination(accounts.Length))
  });
}

public Receive WaitingForTermination(int counter) {
  var remaining = counter;
  return message => message.Match()
    .With<Terminated>(terminated => {
        if(terminated.ActorRef.Path.Name.Contains("account"))
          remaining--;
        if(remaining == 0)
          AllAccountsTerminated();
    })
    .WasHandled;
}
Christian Duhard
@cduhard
Apr 04 2016 18:59
yeah, we were almost identical except I would just poll which is shite, but i was ok with it. My solution requires no changes to the Persistent actor with the down side of a timer
@Horusiath thanks a lot though. I am the only one doing message based work here, I have no one to bounce this stuff off of
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:02
no problem :)
Christian Duhard
@cduhard
Apr 04 2016 19:05
do you have tests for that code?
j/k ;)
@Horusiath regarding your sharding persistence app, the intention is for you sagas to continue on restart correct?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:06
at some point, once it will be finished
Christian Duhard
@cduhard
Apr 04 2016 19:07
that's cool, i wanted to do that, but didn't have the time to reason through it. your sample makes it straightforward
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:08
sagas are hard once you apply "what if something will crash at any point in time" approach
Christian Duhard
@cduhard
Apr 04 2016 19:09
yes they are, that's where I said, maybe later
can I use become from ActorBase?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:10
yes
Christian Duhard
@cduhard
Apr 04 2016 19:11
Used the same way? I only have an override of Receive.
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:11
Context.Become with Receive delegate
Christian Duhard
@cduhard
Apr 04 2016 19:14
Context.Become(WaitingForTermination(accounts.Length));
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:14
yes
but if your have actor based on ReceiverActor or it's persistent version, you won't need much to adjust the sample to work with
Christian Duhard
@cduhard
Apr 04 2016 19:16
is there any reason to use this style over Match?
                else if (message is Shutdown)
                {        
            var m = message as Shutdown;
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:16
no, just convinience
Christian Duhard
@cduhard
Apr 04 2016 19:17
i just left some of it from your original sample. I much prefer Match syntax
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:17
with C# 7 we'll have pattern matching, and all of this will be part of language syntax
Christian Duhard
@cduhard
Apr 04 2016 19:18
i was just going to say we need pattern matching
not sure why they didn;t include it in 6
so C# is becoming scala with readable syntax
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:19
still far from it ;) less expressive type system, no macros etc. also less "magic" ;)
Christian Duhard
@cduhard
Apr 04 2016 19:20
i don't like scala syntax
it feels like a mess
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:21
some parts are nice, some are wtf
like with almost any language
Christian Duhard
@cduhard
Apr 04 2016 19:22
the nice things about C# is they've done a pretty good job adding cool language features relatively cleanly
some better shorthand for list manipulation would be nice
they should also alias standard things like map, fold etc
Christian Duhard
@cduhard
Apr 04 2016 19:35
This message was deleted
ignore that
Become drrr
to11mtm
@to11mtm
Apr 04 2016 19:43
@Horusiath : True, but sometimes guidelines from a library help with Bureaucracy =)
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:49
@to11mtm true, we wanted to warn people using default serializer, that's why from the next release if you have json.net serializer configured as default, you'll receive warning, that it's going to be obsoleted after v1.5
I feel bad about rellying on default serializer for persistence. Some persistence libs already avoided that, providing their own defaults
to11mtm
@to11mtm
Apr 04 2016 19:51
...speaking of... are there any Oracle Persistence libraries out there that anyone is aware of?
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 19:52
Christian Duhard
@cduhard
Apr 04 2016 20:25
@Horusiath EventStore requires json if you want to use the built in projection framework
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 20:34
@cduhard you can always use custom json provider, the one akka uses is quite heavily oriented for akka-specific needs
Christian Duhard
@cduhard
Apr 04 2016 20:34
it's a little late for that :)
we've got 100's of millions of events in the eventstore using the current default serializer
Bartosz Sypytkowski
@Horusiath
Apr 04 2016 20:36
nice to hear - I don't know it eventstore provider doesn't use its own json configuration thou
Christian Duhard
@cduhard
Apr 04 2016 20:46
there isn't a well supported version
i am sorta maintaining my own version
eventstore doesn't require json. if you want to use the native projection engine (v8) is has to be able to parse your events
i think they mentioned supporting a pluggable serializer at some point. MsgPack or proto-buf would be nice
ilhadad
@ilhadad
Apr 04 2016 22:14
I have a receive actor that has both a ReceiveAny and Unhandled method. I am getting in the console log Got unhandled message From:akka://all-system/. Not sure why the message is not delivered.
Yin Zhang
@melcloud
Apr 04 2016 22:15
@ilhadad can you provide your code example and akka config?
ilhadad
@ilhadad
Apr 04 2016 22:15
sure here it is...
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <sectionGroup name="couchbaseClients">
      <section name="couchbase" type="Couchbase.Configuration.Client.Providers.CouchbaseClientSection, Couchbase.NetClient" />
    </sectionGroup>
    <section name="akka" type="Akka.Configuration.Hocon.AkkaConfigurationSection, Akka" />
  </configSections>

  <startup> 
      <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" />
  </startup>

  <akka>
    <hocon>
      <![CDATA[

                akka {
                    # here we are configuring log levels
                    log-config-on-start = off
                    stdout-loglevel = DEBUG
                    loglevel = DEBUG

          // Define an Nlog logger for the Akka system
          loggers = ["Akka.Logger.NLog.NLogLogger, Akka.Logger.NLog"]

            actor {
              provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
            }

            remote {
                            log-remote-lifecycle-events = DEBUG
                            log-received-messages = on

            helios.tcp {
                                transport-class = "Akka.Remote.Transport.Helios.HeliosTcpTransport, Akka.Remote"
                                applied-adapters = []
                                transport-protocol = tcp
                                #will be populated with a dynamic host-name at runtime if left uncommented
                                #public-hostname = "POPULATE STATIC IP HERE"
                                hostname = "127.0.0.1"
                                port = 8777
              }
            }

            cluster {
              seed-nodes = ["akka.tcp://SSAActorSystem@127.0.0.1:4053"]
              #role = ["httpbridge"]

              auto-down-unreachable-after = 30s
            }



        akka.persistence{
                    journal{
                        plugin = "akka.persistence.journal.couchbase"
                        couchbase:{
                            class = "Akka.Persistence.CouchBase.Journal.CouchBaseDbJournal, Akka.Persistence.CouchBase"
                            ServersURI:[
                                "http://127.0.0.1:8091"
                            ],
                            BucketName = "testakka",
                            BucketUseSsl = false,
                            Password = "",
                            DefaultOperationLifespan = 2000,
                            PoolConfiguration.MaxSize = 10,
                            PoolConfiguration.MinSize = 5,
                            SendTimeout = 12000
                        }
                    }

                    snapshot-store{
                        plugin = "akka.persistence.snapshot-store.couchbase"
                        couchbase:{
                            class = "Akka.Persistence.CouchBase.Snapshot.CouchBaseDbSnapshotStore, Akka.Persistence.CouchBase"
                            ServersURI:[
                                "http://127.0.0.1:8091"
                                ],
                                BucketName = "testakka",
                                BucketUseSsl = false,
                                Password = "",
                                DefaultOperationLifespan = 2000,
                                PoolConfiguration.MaxSize = 10,
                                PoolConfiguration.MinSize = 5,
                                SendTimeout = 12000
                            }
                        }
                    }  
                }

            ]]>
    </hocon>
  </akka>

  <couchbaseClients>
    <couchbase useSsl="false">
      <servers>
        <add uri="http://127.0.0.1:8091"></add>
      </servers>
      <buckets>
        <!--<add name="default" useSsl="false" password="" operationLifespan="2000">
                    <connectionPool name="custom" maxSize="10" minSize="5" sendTimeout="12000"></connectionPool>
                </add>-->
        <add name="testakka" useSsl="false" password="" operationLifespan="2000">
          <connectionPool name="custom" maxSize="10" minSize="5" sendTimeout="12000"></connectionPool>
        </add>
      </buckets>
    </couchbase>
  </couchbaseClients>

  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-8.0.0.0" newVersion="8.0.0.0" />
      </dependentAssembly>
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.Owin" publicKeyToken="31bf3856ad364e35" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-3.0.1.0" newVersion="3.0.1.0" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
</configuration>
To add more color to my help request. The actor in question is residing in the node and not remotely.
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:17
which message showed up in Unhandled?
ilhadad
@ilhadad
Apr 04 2016 22:18
It did not call it.
or the receiveany
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:18
can you spell it out for us?
which message, which actor, and when
if it's a big complicated user-defined actor, NBD - but put some meat on the bones
also - which version of Akka.NET? latest stable?
ilhadad
@ilhadad
Apr 04 2016 22:25

Latest akka from nuget.
here is the call to the actor...

            SSAActorSystem.ActorReferences.HTTPClientBridge.Tell(new HTTPSourcedCommand(c, Context.User.Identity.Name, Context.ConnectionId), ActorRefs.Nobody);

Here is how i instantiate the actor...

            //Instantiate the HTTPClient BridgeActor
            ActorReferences.HTTPClientBridge = _ActorSystem.ActorOf(
                Props.Create(() => new HTTPClientBridgeActor(_HTTPClientEventPusher, new IActorRef[] { ActorReferences.ClientList })),
                "HTTPClientBridge"
                );

Here is the actor itself...


    public class HTTPClientBridgeActor:ReceiveActor
    {
        private ILoggingAdapter _logger = Context.GetLogger();

        private static string _ActorType = typeof(HTTPClientBridgeActor).Name;


        public HTTPClientBridgeActor(IHTTPClientInterface httpClientEventInterface, IActorRef[] Supervisors) 
        {

            Become(Initializing);
        }

        #region Actor States

        /// <summary>
        /// This method sets the actor so that it can "boot" itself up.
        /// </summary>
        private void Initializing()
        {
            _logger.Debug("Actor {0} is initializing.", _ActorType);

            // This catch all will log if there are any weird unhandled messages.
            Receive<object>(o =>
            {
                _logger.Debug("{1} Got unhandled message From:{0}", Sender.Path.ToStringWithAddress(),_ActorType);
            });

            _logger.Debug("Actor {0} is Initialized, moving to Ready state.", _ActorType);

            Become(Ready);

        }

        /// <summary>
        /// The command processing state sets this actor ready to receive any command from other actors.
        /// </summary>
        private void Ready()
        {
            _logger.Debug("Actor {0} is getting Ready.", _ActorType);

            // Process Commands that are coming from external clients
            Receive<HTTPSourcedCommand>(c => ProcessExternalHTTPClientCommandHandler(c));
            Receive<string>(s=> HandleStringCommand(s));

            // This catch all will log if there are any weird unhandled messages.
            ReceiveAny(o =>
            {
                _logger.Debug("{1} got unhandled message from:{0} Unhandled Message:{1}", Sender.Path.ToStringWithAddress(), o.GetType().Name, _ActorType);
            });


            _logger.Debug("Actor {0} is Ready.", _ActorType);

        }

        #endregion Actor States


        protected override void Unhandled(object message)
        {
            //Do something with the message.
        }
... more code after this
Here is the message that is being sent...
    public class HTTPSourcedCommand
    {
        public HTTPSourcedCommand()
        {

        }

        public HTTPSourcedCommand(string commandType, string area, string id = null, string fieldName = null, object data = null, string user = null)
        {
            CommandType = commandType;
            Data = data;
            Area = area;
            FieldName = fieldName;
            Id = id;
            User = user;
        }

        /// <summary>
        /// Returns a copy of the original command with the user name added.
        /// </summary>
        /// <param name="originalCommand"></param>
        /// <param name="user">user name</param>
        public HTTPSourcedCommand( HTTPSourcedCommand originalCommand,string user,string connectionId)
        {
            CommandType = originalCommand.CommandType;
            Data = originalCommand.Data;
            Area = originalCommand.Area;
            FieldName = originalCommand.FieldName;
            Id = originalCommand.Id;
            User = user;
            ConnectionId = connectionId;
        }


        [JsonProperty]
        public string CommandType { get; private set; }
        [JsonProperty]
        public object Data { get; private set; }
        [JsonProperty]
        public string Area { get; private set; }
        [JsonProperty]
        public string FieldName { get; private set; }
        [JsonProperty]
        public string Id { get; private set; }
        [JsonProperty]
        public string User { get; private set; }
        [JsonProperty]
        public string ConnectionId { get; private set; }

    }
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:27
Become call in the constructor
there's your problem
just call Initializing
we had an issue logged about that somewhere
but in general - don't call Become for behavior #1
only call it when switching from one active behavior to another
there's stuff inside the behavior stack still being initialized when the actor instance is being constructed
ilhadad
@ilhadad
Apr 04 2016 22:29
It seemed to call the Initializing method.
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:29
so what behavior is the actor in when the issue occurs?
initializing or ready
ilhadad
@ilhadad
Apr 04 2016 22:29
Ready
Initializing is called and then it calls ready.
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:30
we might have fixed the Become in the constructor bug then
and so ReceiveAny misses a message?
ilhadad
@ilhadad
Apr 04 2016 22:31
Here's the console output...
2016-04-04 18:10:16.6303|INFO|EY.SSA.ActorSystemBridge.Program|Starting SSA Actor System at HTTP Client Bridge at:CF_IHadad
[DEBUG][4/4/2016 10:10:16 PM][Thread 0008][EventStream] StandardOutLogger started
[INFO][4/4/2016 10:10:16 PM][Thread 0015][[akka://SSAActorSystem/system/log1-NLogLogger]] NLogLogger started
[DEBUG][4/4/2016 10:10:16 PM][Thread 0008][EventStream(SSAActorSystem)] Logger log1-NLogLogger [NLogLogger] started
[DEBUG][4/4/2016 10:10:16 PM][Thread 0008][EventStream(SSAActorSystem)] StandardOutLogger being removed
2016-04-04 18:10:16.9651|INFO|EY.SSA.ActorSystemBridge.Program|Started SSA Actor System at HTTP Client Bridge at:CF_IHadad
2016-04-04 18:10:36.2219|DEBUG|EY.SSA.CommonBusinessLogic.Actors.HTTPClientBridgeActor|Actor HTTPClientBridgeActor is initializing.
2016-04-04 18:10:38.6722|DEBUG|EY.SSA.CommonBusinessLogic.Actors.HTTPClientBridgeActor|Actor HTTPClientBridgeActor is Initialized, moving to Ready state.
2016-04-04 18:10:38.6922|INFO|EY.SSA.ActorSystemBridge.Program|HTTP Server running on http://localhost:8080
2016-04-04 18:10:38.7037|DEBUG|EY.SSA.CommonBusinessLogic.Actors.HTTPClientBridgeActor|Actor HTTPClientBridgeActor is getting Ready.
Enter a command:
2016-04-04 18:10:38.7197|DEBUG|EY.SSA.CommonBusinessLogic.Actors.HTTPClientBridgeActor|Actor HTTPClientBridgeActor is Ready.
2016-04-04 18:10:49.7208|DEBUG|EY.SSA.CommonBusinessLogic.Actors.HTTPClientBridgeActor|HTTPClientBridgeActor Got unhandled message From:akka://all-systems/
2016-04-04 18:10:49.7354|DEBUG|EY.SSA.CommonBusinessLogic.Actors.HTTPClientBridgeActor|HTTPClientBridgeActor Got unhandled message From:akka://all-systems/
Yep. it is as if the actor never received it.
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:32
ok, so it looks like the Receive<object> you set in the `Initialzing state is still there
because that's your own debug message showing up
ilhadad
@ilhadad
Apr 04 2016 22:32
Oh yeah
I see that.
wierd.
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:33
I'm wondering if this did something funny to the first behavior
ilhadad
@ilhadad
Apr 04 2016 22:33
So should I just call Initializing from the constructor?
just to test?
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:33
like it basically did INITIALIZING + READY --> your behaviuor
give it a try
ilhadad
@ilhadad
Apr 04 2016 22:33
ok
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:33
calling Become twice is essentially what's happening here
not sure what our test coverage is for that
ilhadad
@ilhadad
Apr 04 2016 22:34
That was it!!!!
It sounds like it preferred the first become over the second one.
It's a good thing I was not consistent in my logging messages :smile:
Thanks so much. We've been scratching our heads for a bit here.
Aaron Stannard
@Aaronontheweb
Apr 04 2016 22:44
filed #1849 for you @ilhadad
joonhwan
@joonhwan
Apr 04 2016 22:47
I have to actor system remotely associated, and if I make one of them crash, the other detect actor death in > ~30s after detect endpoint dissociation.
Is there any way for me to make it detect in a fewer delay?
(please read following log time stamp)
2016/04/05 07:22:43.637   109 ERROR Akka.Actor.OneForOneStrategy  Disassociated Akka.Remote.EndpointDisassociatedException: Disassociated
2016/04/05 07:22:43.637   109  WARN Akka.Remote.ReliableDeliverySupervisor  Association with remote system akka.tcp://rcsLocalSystem@203.239.173.241:17001 has failed; address is now gated for 5000 ms. Reason is: [Akka.Remote.EndpointDisassociatedException: Disassociated
2016/04/05 07:23:06.917   159  WARN Akka.Remote.RemoteWatcher  Detected unreachable: [akka.tcp://rcsLocalSystem@203.239.173.241:17001] 
2016/04/05 07:23:06.917   159  WARN Akka.Event.DummyClassForStringSources  Association to [akka.tcp://rcsLocalSystem@203.239.173.241:17001] having UID [1302300886] is irrecoverably failed. UID is now quarantined and all messages to this UID will be delivered to dead letters. Remote actorsystem must be restarted to recover from this situation.
you can turn that heartbeat pause way down
or swap out the failure detector implementation with something simpler like a deadline failure detector
which the transport failure detector (immediately previous node in the configuration) does
I personally don't like the PhiAccrualFailureDetector - it's meant to be adaptive based around latency
but I think it's one of those things that sounds great in theory but just makes your system more difficult to debug
DeadlineFailureDetector is pretty clear
joonhwan
@joonhwan
Apr 04 2016 23:30
@Aaronontheweb thanks. wow. You do not like that default failure detector. you know what, any guys like me who've started using Akka.net, the comments in the default Remote.conf is very 'reference' material or something, and it makes me feel that PhiAccrualFailureDetector is something guru's guidance(it even contains the reference paper link!). :smile:
Anyway I'll try that detector. Thanks.