These are chat archives for akkadotnet/akka.net

9th
Jun 2016
Aaron Stannard
@Aaronontheweb
Jun 09 2016 06:41 UTC
@alexvaluyskiy getting a lot closer on the ActorCell bits
in the middle of reworking that and the dispatcher stuff
dispatchers and mailboxes worked backwards in Akka.NET compared to how they're intended to work on the JVM - have that fixed but plugging it all back in together in the ActorCell is slow going. Needed to get that right first so the way we handle orderly shutdown and system message invocation is consistent.
I'm interested to see what the benchmarks show - it should be a lot easier to squeeze out a bunch of extra performance out of this architecture compared to the previous one
making improvements inside how the dispatchers work will have a much more noticeable impact than they did previously
frasermolyneux
@frasermolyneux
Jun 09 2016 14:27 UTC
Hi there, has anyone been successful with a reliable Azure cluster deployment with the classic cloud services?
Peter Bergman
@peter-bannerflow
Jun 09 2016 16:28 UTC
@frasermolyneux I would like to know that as well :)
frasermolyneux
@frasermolyneux
Jun 09 2016 16:28 UTC
I managed to get it working.. for 30 seconds then it crapped out with association errors
Peter Bergman
@peter-bannerflow
Jun 09 2016 16:29 UTC
Yup, same here
frasermolyneux
@frasermolyneux
Jun 09 2016 16:29 UTC
I believe its to do with the different types of endpoints, so currently trying it on internal endpoint tyes only
Peter Bergman
@peter-bannerflow
Jun 09 2016 16:30 UTC
Yeah, I have only internal endpoints for node to node communication within the cluster, then I have one external endpoint to one role which is a web role
frasermolyneux
@frasermolyneux
Jun 09 2016 16:31 UTC
So is your web role using the actor system?
Peter Bergman
@peter-bannerflow
Jun 09 2016 16:32 UTC
Yes, it acts as a endpoint for external HTTP request, then it communicates with other actors within the system
frasermolyneux
@frasermolyneux
Jun 09 2016 16:33 UTC
OK I get it - so generally as long as all the actors are using internal endpoints it should be fine
Peter Bergman
@peter-bannerflow
Jun 09 2016 16:33 UTC
Not sure, I am using internal endpoints only for actor to actor communication within the system but that still causes issues
frasermolyneux
@frasermolyneux
Jun 09 2016 16:34 UTC
Would you mind showing me your EndPoints section from your ServiceDefinition.csdef?
Peter Bergman
@peter-bannerflow
Jun 09 2016 16:34 UTC
Sure
frasermolyneux
@frasermolyneux
Jun 09 2016 16:34 UTC
Thanks :)
Peter Bergman
@peter-bannerflow
Jun 09 2016 16:37 UTC
Hmm I'll send it in a private msg
Aaron Stannard
@Aaronontheweb
Jun 09 2016 20:24 UTC
@frasermolyneux @peter-bannerflow so I've done some work with Akka.Cluster on classic cloud services before
I don't recommend it - the issue is that their deployment model as-is is too simple
it's never very feasible to update just one service
or do it in a way, if that service is stateful, in a manner that won't disrupt the entire cluster
cloud services were really designed for stateless web applications
I've seen some successful service fabric deployments
and I'
I'm personally going to be experimenting with ARM
but you know what - I didn't spend that much time on
it
so maybe it's possible to get it right
Aaron Stannard
@Aaronontheweb
Jun 09 2016 21:14 UTC
@Danthar lol glad you liked it
Aaron Stannard
@Aaronontheweb
Jun 09 2016 21:24 UTC
@alexvaluyskiy FYI, I'm making some minor changes to routers now as part of the mailbox stuff
Resizers and RoutedActorRef changes mostly - pretty minor
@Aaronontheweb Do you have a recommendation for what works well for clustering in Azure?
Aaron Stannard
@Aaronontheweb
Jun 09 2016 21:27 UTC
@qwoz AWS
haha
Aaron Stannard
@Aaronontheweb
Jun 09 2016 21:27 UTC
lol
I'll be doing some deployment and DevOps guides after we ship 1.1
so I
I'll do some practical research there
and see what I can come up with
last pass I took with Azure was using Cloud Services
about a year ago
I'm going to be deploying some Akka.Cluster stuff on Azure this year that Petabridge's business runs on
so I'd better get it right haha
ilhadad
@ilhadad
Jun 09 2016 21:44 UTC
If the persistence actor does not get any messages back from db and I immediately get RecoveryCompleted could it still receive messages other messages? Something freaking weird is going on. I cannot get the persistence actor to receive any messages.
Aaron Stannard
@Aaronontheweb
Jun 09 2016 21:45 UTC
@ilhadad not sure, but ccing @cconstantin @Horusiath
Bartosz Sypytkowski
@Horusiath
Jun 09 2016 21:45 UTC
@ilhadad do you have any logs?
ilhadad
@ilhadad
Jun 09 2016 21:46 UTC
Yes, I can see it receiving the recovery message and then Becoming into the state where I can receive "command" messages
I can paste some code here. Give me a moment
Constructor...
public ClientListActor()
{
//Set up the command handlers
UseRecoveryCommandHandler = new Dictionary<string, Action<Command>>();
        // Set up a state object
        _ActorState = new ClientListState(PersistenceId, _ActorType);

        // Put actor in recovering state
        Recovering();
    }
I cannot figure out to make the code look good
Bartosz Sypytkowski
@Horusiath
Jun 09 2016 21:49 UTC
tripple ` at beginning and end line
ilhadad
@ilhadad
Jun 09 2016 21:50 UTC
``` private void Recovering()
{
_logger.Debug($"Setting Up Recovery with persistence id: {PersistenceId}");
        Recover<ClientIdGetListRequest>(r =>
        {
            _logger.Debug($"Recovering stashing message - not ready. from:{Sender.Path.ToStringWithAddress()} Message:{r.GetType().Name}");
            Stash.Stash();
        });

        // Switch state to CommandProcessing to process other actor commands.
        Recover<RecoveryCompleted>(rc =>
        {
            _logger.Info("Recovery Complete.", Self.Path.ToStringWithAddress());
            Become(CommandProcessing);
            // Now that we are fully recovered let me process all the messages that came in.
            Stash.UnstashAll();

        });

        // ******** IMPORTANT ***********
        // For each command there should be a handler defined
        // ******************************
        UseRecoveryCommandHandler.Add("ClientListInsertCommand", command => InsertNewClientListItemRecoveryCommand(command as ClientListInsertCommand));
        UseRecoveryCommandHandler.Add("ClientListUpdateCommand", command => UpdateClientListRecoveryCommand(command as ClientListUpdateCommand));
        UseRecoveryCommandHandler.Add("ClientListDeleteCommand", command => DeleteClientListRecoveryCommand(command as ClientListDeleteCommand));
        UseRecoveryCommandHandler.Add("ClientListUnDeleteCommand", command => UnDeleteClientListRecoveryCommand(command as ClientListUnDeleteCommand));

        // Process any snapshots recovered from the data store
        Recover<SnapshotOffer>(snapshotOffer => ProcessSnapshot(snapshotOffer));

        // These will be commands restored from the data store.
        Recover<JObject>(jo =>
        {
            //Console.WriteLine("Received object: " + jo);
            try
            {
                string commandName = jo["CommandClass"].Value<string>();
                Type commandType = Type.GetType(typeof(Command).Namespace + "." + commandName);
                Command cmd = jo.ToObject(commandType) as Command;
                UseRecoveryCommandHandler[commandName](cmd);
            }
            catch (Exception e)
            {
                _logger.Error("Error:{0}\nFailed to process journal entry:{1}", e.Message, jo);
            }
        });

        // Restores Configuration
        Recover<SetSnapshotTriggerCount>(c => { _SnapshotTriggerCount = c.Value; });
        Recover<SetInactivityFlushSec>(c => { _InactivityFlushSec = c.Value; });

        Recover<object>(message =>
        {
            Stash.Stash();
            _logger.Debug($"During Recovery stashing unhandled message from:{Sender.Path.ToStringWithAddress()} Got Unhandled Message:{message.GetType().Name}");
        });

        _logger.Debug($"Set Up Recovery with persistence id: {PersistenceId}");

    }
This the state where I am supposed to get other messages.
Here is the final state
``` private void CommandProcessing()
{
_logger.Info("Getting Ready.");
// Commands
Command<SaveSnapshotSuccess>(c => HandleSuccessfulSnapshotSave(c));
Command<SaveSnapshotFailure>(c => HandleUnSuccessfulSnapshotSave(c));
        // Events
        Command<ClientInsertedEvent>(e => HandleClientInsertedEvent(e));
        Command<ClientDeletedEvent>(e => HandleClientDeletedEvent(e));
        Command<ClientUnDeletedEvent>(e => HandleClientUnDeletedEvent(e));
        Command<ClientUpdatedEvent>(e => HandleClientUpdatedEvent(e));
        Command<SubscribedForCommandEvents>(e => { _logger.Info("Now listening to:{0}",e.Id); });

        // Requests
        Command<ClientGetStateRequest>(r => { Sender.Tell(new ClientGetStateResponse(Sender, null, r)); });
        Command<ClientIdGetListRequest>(r => HandleClientIdGetListRequest(r));

        // Configuration
        Command<SetSnapshotTriggerCount>(c => { Persist<SetSnapshotTriggerCount>(c, SetSnapshotTriggerConfigurationValue); });
        Command<SetInactivityFlushSec>(c => { Persist<SetInactivityFlushSec>(c, SetInactivityFlushSecConfigurationValue); });

        // Handle any string commands
        Command<string>(s => HandleStringCommand(s));

        // This catch all will log if there are any weird unhandled messages.
        Command<object>(message =>
        {
            _logger.Debug($"In Command State Received unhandled message from:{Sender.Path.ToStringWithAddress()} Unhandled Message:{message.GetType().Name}");
        });

        _logger.Debug("Ready.");
    }
When the 'RecoveryCompleted' message is received is supposed to go into the "CommandProcessing" state. There it is supposed to receive other messages. The thing is that I don't get jack!
I am pissed.
I even tried sending multiple messages spaced out in time thinking it was a racing condition. Like this...
        // Request the client list to instantiate all the client actors
        Context.System.Scheduler.ScheduleTellOnce(10000, Context.Child(_ClientListPersistenceId), new ClientIdGetListRequest(Self), Self);
        Context.System.Scheduler.ScheduleTellOnce(1000, cl, new ClientIdGetListRequest(Self), Self);
        Context.System.Scheduler.ScheduleTellOnce(2000, cl, new ClientIdGetListRequest(Self), Self);
        Context.System.Scheduler.ScheduleTellOnce(3000, cl, new ClientIdGetListRequest(Self), Self);
        Context.System.Scheduler.ScheduleTellOnce(4000, cl, new ClientIdGetListRequest(Self), Self);
        Context.System.Scheduler.ScheduleTellOnce(5000, cl, new ClientIdGetListRequest(Self), Self);
        Context.System.Scheduler.ScheduleTellOnce(6000, cl, new ClientIdGetListRequest(Self), Self);
        Context.System.Scheduler.ScheduleTellOnce(1000, cl, "test", Self);
        Context.System.Scheduler.ScheduleTellOnce(2000, cl, "test", Self);
        Context.System.Scheduler.ScheduleTellOnce(3000, cl, "test", Self);
        Context.System.Scheduler.ScheduleTellOnce(4000, cl, "test", Self);
        Context.System.Scheduler.ScheduleTellOnce(5000, cl, "test", Self);
        Context.System.Scheduler.ScheduleTellOnce(6000, cl, "test", Self);
where cl is an IActorRef
Bartosz Sypytkowski
@Horusiath
Jun 09 2016 21:55 UTC
This message was deleted
ilhadad
@ilhadad
Jun 09 2016 21:56 UTC
2016-06-09 17:55:31.2188|INFO|EY.SSA.ActorSystemHost.Program|Version:C# Build: 1.0.0.0
2016-06-09 17:55:31.3248|INFO|EY.SSA.ActorSystemHost.Program|Starting SSA Actor System at ActorSystemHost at:CF_IHadad
[DEBUG][6/9/2016 9:55:31 PM][Thread 0009][EventStream] StandardOutLogger started
[INFO][6/9/2016 9:55:31 PM][Thread 0013][[akka://SSAActorSystem/system/log1-NLogLogger]] NLogLogger started
[DEBUG][6/9/2016 9:55:31 PM][Thread 0009][EventStream(SSAActorSystem)] Logger log1-NLogLogger [NLogLogger] started
[DEBUG][6/9/2016 9:55:31 PM][Thread 0009][EventStream(SSAActorSystem)] StandardOutLogger being removed
2016-06-09 17:55:31.6709|INFO|EY.SSA.ActorSystemHost.Program|Started SSA Actor System at ActorSystemHost at:CF_IHadad
2016-06-09 17:55:31.6854|INFO|EY.SSA.ActorSystemHost.Program|Instantiating SupervisorRegistryActor
2016-06-09 17:55:31.7049|INFO|EY.SSA.ActorSystemHost.Program|Instantiated SupervisorRegistryActor
2016-06-09 17:55:31.7049|DEBUG|EY.SSA.CommonBusinessLogic.Actors.SupervisorRegistryActor|Initializing.
2016-06-09 17:55:31.7205|INFO|EY.SSA.ActorSystemHost.Program|Instantiating Client Supervisor.
2016-06-09 17:55:31.7205|DEBUG|EY.SSA.CommonBusinessLogic.Actors.SupervisorRegistryActor|Getting Ready.
2016-06-09 17:55:31.7330|INFO|EY.SSA.ActorSystemHost.Program|Instantiated Client Supervisor.
2016-06-09 17:55:31.7330|DEBUG|EY.SSA.CommonBusinessLogic.Actors.SupervisorRegistryActor|Ready.
2016-06-09 17:55:31.7490|INFO|EY.SSA.ActorSystemHost.Program|Host Actor System running on EY.SSA.ActorSystemHost.vshost.exe
2016-06-09 17:55:31.7490|DEBUG|EY.SSA.CommonBusinessLogic.Actors.SupervisorRegistryActor|Initialized.
Enter a command:
2016-06-09 17:55:31.7675|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Is registering.
2016-06-09 17:55:31.7770|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Waiting for Registry Supervisor.
2016-06-09 17:55:35.1723|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Registered
2016-06-09 17:55:35.1863|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Initializing.
2016-06-09 17:55:35.2023|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Instantiating ClientList actor.
2016-06-09 17:55:35.2123|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Instantiated ClientList actor.
2016-06-09 17:55:35.2123|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Instantiating ClientStateAccumulator actor.
2016-06-09 17:55:35.2353|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Instantiated ClientStateAccumulator actor.
2016-06-09 17:55:35.2453|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientStateAccumulator|Initializing.
2016-06-09 17:55:35.2453|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientStateAccumulator|Initialized.
2016-06-09 17:55:35.2648|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Initialized.
2016-06-09 17:55:35.2748|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientSupervisor|Received message ClientGetChildActorRefs. Stashing not ready to handle.
2016-06-09 17:55:35.2748|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Setting Up Recovery with persistence id: ClientList
2016-06-09 17:55:35.2930|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Set Up Recovery with persistence id: ClientList
2016-06-09 17:55:35.7392|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Recovery Complete.
2016-06-09 17:55:35.7507|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Getting Ready.
2016-06-09 17:55:35.7617|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Ready.
2016-06-09 17:55:37.5751|DEBUG|EY.SSA.CommonBusinessLogic.Actors.SupervisorRegistryActor|Sending list of supervisors to:akka.tcp://SSAActorSystem@127.0.0.1:50742/user/HTTPClientBridge
log messages
You can see at the end how it goes through the states (ClientListActor). Goes from Setting Up Recovery->Set Up Recovery->Recovery Complete->Getting Ready->Ready. Some message should have been handled after that but it is as if nothing is coming in.
Bartosz Sypytkowski
@Horusiath
Jun 09 2016 22:02 UTC
@ilhadad
  1. You don't need to change behavior for Recovery - it works only for Commands.
  2. Recover can be triggered only by persistent journal or snapshot store. All of your messages send to an actor always will trigger Command methods.
  3. Recovery is used only for recovering actor's state from events/snapshots - so there is no sense in having Recover<ClientIdGetListRequest> as this denotes a command, not an event.
  4. For the same reason there is no need to use stashing inside recovery - actor won't process any commands until it will restore its state anyway.
  5. If you have to use try/catch inside Recover, your actor is most probably already broken.
ilhadad
@ilhadad
Jun 09 2016 22:04 UTC
So what happens if it receives a message before recoverycomplete?
does it stash it?
Bartosz Sypytkowski
@Horusiath
Jun 09 2016 22:05 UTC
yes, automatically
ilhadad
@ilhadad
Jun 09 2016 22:05 UTC
and unstash it automatically?
ok
ilhadad
@ilhadad
Jun 09 2016 22:28 UTC
Changed it so that everything is set up at once. Here's the code:
``` private void Recovering()
{
_logger.Debug($"Setting Up Persistence Actor with persistence id: {PersistenceId}");
UseRecoveryCommandHandler.Add("ClientListInsertCommand", command => InsertNewClientListItemRecoveryCommand(command as ClientListInsertCommand));
UseRecoveryCommandHandler.Add("ClientListUpdateCommand", command => UpdateClientListRecoveryCommand(command as ClientListUpdateCommand));
UseRecoveryCommandHandler.Add("ClientListDeleteCommand", command => DeleteClientListRecoveryCommand(command as ClientListDeleteCommand));
UseRecoveryCommandHandler.Add("ClientListUnDeleteCommand", command => UnDeleteClientListRecoveryCommand(command as ClientListUnDeleteCommand));
        // Process any snapshots recovered from the data store
        Recover<SnapshotOffer>(snapshotOffer => ProcessSnapshot(snapshotOffer));

        // These will be commands restored from the data store.
        Recover<JObject>(jo =>
        {
            //Console.WriteLine("Received object: " + jo);
            try
            {
                string commandName = jo["CommandClass"].Value<string>();
                Type commandType = Type.GetType(typeof(Command).Namespace + "." + commandName);
                Command cmd = jo.ToObject(commandType) as Command;
                UseRecoveryCommandHandler[commandName](cmd);
            }
            catch (Exception e)
            {
                _logger.Error("Error:{0}\nFailed to process journal entry:{1}", e.Message, jo);
            }
        });

        Recover<RecoveryCompleted>(rc =>
        {
            _logger.Info("Recovery Complete.", Self.Path.ToStringWithAddress());
        });

        try
        {
            _logger.Info("Setting Up Command Handlers.");
            // Commands
            Command<SaveSnapshotSuccess>(c => HandleSuccessfulSnapshotSave(c));
            Command<SaveSnapshotFailure>(c => HandleUnSuccessfulSnapshotSave(c));

            // Events
            Command<ClientInsertedEvent>(e => HandleClientInsertedEvent(e));
            Command<ClientDeletedEvent>(e => HandleClientDeletedEvent(e));
            Command<ClientUnDeletedEvent>(e => HandleClientUnDeletedEvent(e));
            Command<ClientUpdatedEvent>(e => HandleClientUpdatedEvent(e));
            Command<SubscribedForCommandEvents>(e => { _logger.Info("Now listening to:{0}", e.Id); });

            // Requests
            Command<ClientGetStateRequest>(r => { Sender.Tell(new ClientGetStateResponse(Sender, null, r)); });
            Command<ClientIdGetListRequest>(r => HandleClientIdGetListRequest(r));

            // Configuration
            Command<SetSnapshotTriggerCount>(c => { Persist<SetSnapshotTriggerCount>(c, SetSnapshotTriggerConfigurationValue); });
            Command<SetInactivityFlushSec>(c => { Persist<SetInactivityFlushSec>(c, SetInactivityFlushSecConfigurationValue); });

            // Handle any string commands
            Command<string>(s => HandleStringCommand(s));

            // This catch all will log if there are any weird unhandled messages.
            Command<object>(message =>
            {
                _logger.Debug($"In Command State Received unhandled message from:{Sender.Path.ToStringWithAddress()} Unhandled Message:{message.GetType().Name}");
            });

            _logger.Debug("Command Handlers Set Up.");

        }
        catch (Exception ex)
        {
            _logger.Error("Something went really bad during recovery.");
        }
        _logger.Debug($"Set Up persistence actor persistence id: {PersistenceId}");

    }```
Still no luck
Here's the log
2016-06-09 18:27:02.2246|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Setting Up Persistence Actor with persistence id: ClientList
2016-06-09 18:27:02.2436|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Setting Up Command Handlers.
2016-06-09 18:27:02.2436|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Command Handlers Set Up.
2016-06-09 18:27:02.2656|DEBUG|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Set Up persistence actor persistence id: ClientList
2016-06-09 18:27:05.0709|INFO|EY.SSA.CommonBusinessLogic.Actors.ClientListActor|Recovery Complete.
The try/catch did not trigger. So the code is not in error.