These are chat archives for akkadotnet/akka.net

9th
Dec 2017
wilddeveloperappears
@wilddeveloperappears
Dec 09 2017 01:15
Hi, I'm trying to set the result of Context.GetLogger() at runtime to something different. Is this possible, and how would I do this? I don't want to use the Hocon to do this, because I want the logging infrastructure to do something unique for each ActorSystem that's running.
ryzam
@ryzam
Dec 09 2017 02:00

Hi guys, I'm doing a demo project using akka cluster and facing a problem to send message to the cluster actor. Already run Lighthouse project as Seed-Nodes and I also can deploy the actor remotely and having actor reference but the actor are not able to receiving a message. If i change user-role=api then it can receive a message

class Program
    {
        static void Main(string[] args)
        {
            var config = ConfigurationFactory.ParseString(@"
                akka {
                        actor { 
                            provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster""
                        }

                        remote {
                            log-remote-lifecycle-events = DEBUG
                            dot-netty.tcp {    
                                hostname = ""127.0.0.1""
                                port = 5001
                            }
                        }            

                        cluster {
                            #will inject this node as a self-seed node at run-time
                            seed-nodes = [""akka.tcp://akkademo@127.0.0.1:4053""]
                            roles = [actorserver]
                        }
                    }
            ");
            using (ActorSystem system = ActorSystem.Create("akkademo",config))
            {
                Console.ReadLine();
            }
        }
    }

and here is the API cluster side

    class Program
    {
        static void Main(string[] args)
        {
            var config = ConfigurationFactory.ParseString(@"
                akka {
                        actor { 
                            provider = ""Akka.Cluster.ClusterActorRefProvider, Akka.Cluster""

                            deployment {
                                /actorServerGraphQL  {
                                   router = round-robin-pool
                                   nr-of-instances = 10
                                   cluster {
                                      enabled = on
                                      max-nr-of-instances-per-node = 2
                                      allow-local-routees = on
                                      use-role = actorserver
                                   }
                                }
                            }
                        }

                        remote {
                            log-remote-lifecycle-events = DEBUG
                            dot-netty.tcp {    
                                hostname = ""127.0.0.1""
                                port = 5002
                            }
                        }            

                        cluster {
                            #will inject this node as a self-seed node at run-time
                            seed-nodes = [""akka.tcp://akkademo@127.0.0.1:4053""]
                            roles = [api]
                        }
                    }
            ");
            using (ActorSystem system = ActorSystem.Create("akkademo", config))
            {



                var actorServer = system.ActorOf(Props.Create<ActorServerGraphQL>().WithRouter(FromConfig.Instance), "actorServerGraphQL");


                actorServer.Tell(new QueryDatabase("Start query"));

                //var apiActor = system.ActorOf(Props.Create(() => new ApiActor(actorServer)), "apiActor");


                //apiActor.Tell(new QueryDatabase("Start query"));


                Console.ReadLine();
            }
        }
    }
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 08:20

Do you think this might help ? http://getakka.net/articles/clustering/cluster-sharding.html#retrieving-sharding-state

@jalchr sorry, but I'm out of the context.

jalchr
@jalchr
Dec 09 2017 08:21
@Horusiath regarding the Getting the children for a shard ...
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 08:22
yes, if you need to know about alive entities living in the same shard
I've wrote once a blog post where I've described all query messages, that can be used to communicate with cluster sharding: https://petabridge.com/blog/cluster-sharding-technical-overview-akkadotnet/#getting-shard-metadata
jalchr
@jalchr
Dec 09 2017 08:24
On GetShardRegionState shard region will reply with ShardRegionState containing data about shards living in the current actor system and what entities are alive on each one of them.
On GetClusterShardingStats shard region will reply with ClusterShardingStats having information about shards living in the whole cluster and how many entities alive in each one of them.
okay ... this gives info about "All active Shards" ... not a single one ... right ?
very well ...
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 08:26
@wilddeveloperappears you still can set up a separate HOCON config for each ActorSystem, with different logging in mind. In general it's not possible to setup a separate logging adapter per actor, as logging is running within actor system event stream. It doesn't discriminate different actors.
@jalchr yes, for a single shard, you have corresponding message types described bellow in the same article
jalchr
@jalchr
Dec 09 2017 08:31

How can I query for the stats ? Is it a Tell or Ask ?

 _shard.Ask< ?? >(Shard.GetCurrentShardState.Instance);

or

 _shard.Tell(Shard.GetCurrentShardState.Instance);

and How I can get the response out of a 'Tell' ??

Bartosz Sypytkowski
@Horusiath
Dec 09 2017 08:38
shard.Ask<Shard.CurrentShardState>(Shard.GetCurrentShardState.Instance); - it's described there, by convention query names have Getxxx prefix, while replies don't
with Tell, shard will simply respond to a message sender (an actor)
jalchr
@jalchr
Dec 09 2017 08:41
Oops ... I missed that in the documentation ... strongly appreciated
jalchr
@jalchr
Dec 09 2017 09:21
@Horusiath I'm getting this error:
2017-12-09 11:20:41,542 [33] WARN  Akka.Cluster.Sharding.ShardRegion - Message does not have an extractor defined in shard [FileHandler] so it was ignored: Akka.Cluster.Sharding.Shard+GetCurrentShardState
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 09:23
to which actor do you send that message?
it looks like you're trying to contact with sharded actor instead of shard itself
jalchr
@jalchr
Dec 09 2017 09:25
I think so ... here is what I'm doing:
            _shard = ClusterSharding.Get(Context.System).ShardRegion(nameof(FileHandler));
            var state = _shard.Ask<Shard.CurrentShardState>(Shard.GetCurrentShardState.Instance).Result;
            var exists = state.EntityIds.Contains(newVideo.File.ToLower());
How to talk with the "shard" itself ?
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 10:22
@jachlr in your case _shard is actually ShardRegion. General notion is that:
  • Actors managed by cluster sharding often are referred to as entities
  • Entities are grouped in shards. Entities within the same shard always live on the same machine, when they are rebalanced to another one, they are always rebalanced together. If you want to get reference to target shard, keep in mind that shard is an actor parent for it's entities (so you can use Context.Parent from within entity).
  • Shards themselves are grouped within shard regions. Shard region is a container for all shards of a given actor type living on a target actor system.
jalchr
@jalchr
Dec 09 2017 11:01
@Horusiath I think I understand the terminology. But it would clearer if translated to code.
So _shard is a region.
How do I query for the "shard state" ... ?
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 11:04
lets first define, what information do you really want to get? "shard state" is very broad
jalchr
@jalchr
Dec 09 2017 11:42
Very well, my ApiMaster is trying to launch a new "File Processing" operation .. I'm trying to detect if a 'job' is already running or not
private void HandleFindRunningJob(FindRunningNewVideo newVideo)
        {
            //var state = _shard.Ask<Shard.CurrentShardState>(Shard.GetCurrentShardState.Instance).Result;
            //var exists = state.EntityIds.Contains(newVideo.File.ToLower());

            var haveChild = Context.Child(newVideo.File.ToLower());
            if (haveChild != ActorRefs.Nobody)
            {
                ApiBroadcaster.Tell(new NewVideoFound(newVideo.File));

                _mediator.Tell(new Publish(Topics.Reporting, new ReportStatus(ReportStatusEnum.Info,
                    $"HandleFindRunningJob() : 'already processed' file = {newVideo.File}"
                    )));
                _logger.Info($"HandleFindRunningJob() : 'already processed' file = {newVideo.File}");

            }
        }
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 12:32
is there some kind of 1-1 relation between "job" and a particular actor instance?
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 12:45
usually "job" describes some stateless behavior - cluster sharding has more sense in stateful problems (when you have a logical domain entity like User, Department, Vehicle etc.). You don't need to check if entity is alive, as you shouldn't care about that. If it's running, just queue the next message for it, it will be picked up at some point. If it's not alive it will be created ad-hoc and start processing the message.
jalchr
@jalchr
Dec 09 2017 13:47

I see what you mean. I have FileProcessing to be a stateful problem. It undergoes several steps. I need to make sure I have 1 actor (entity) per file.
Cause now, the duplicate jobs are causing file locking issues (a file is being used by another process) sort of things.

Before I move to persistent actors, I used var haveChild = Context.Child(newVideo.File.ToLower()); ...
Now, I'm using shards and entities ... I need to mimic the same functionality ...
Much Appreciated

Bartosz Sypytkowski
@Horusiath
Dec 09 2017 13:57
@jalchr you can just post message to an entity - it will be created ad-hoc if it wasn't alive. Just keep in mind, that sharded entities are a bit slower and more heavyweight than normal actors (because of all automatic lifecycle and cluster message routing machanisms).
jalchr
@jalchr
Dec 09 2017 14:00
@Horusiath You mean I don't have to query for the shard state to detect whether or not an entity already exists ?
And simply send a message, as long as an entity can exist only once... right ?
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 14:00
yes
jalchr
@jalchr
Dec 09 2017 14:01
very well...
In all cases, how can we query a shard for its 'state' :) ?
jalchr
@jalchr
Dec 09 2017 14:28
I'm experience "shortage" in communication ....
here is a sample log file
2017-12-09 16:03:35,998 [6] WARN  Akka.Cluster.ClusterCoreDaemon - Cluster Node [akka.tcp://ArchiveSystem@140.125.4.1:16568] - Marking node(s) as UNREACHABLE [Member(address = akka.tcp://ArchiveSystem@140.125.4.2:16668, Uid=81451205 status = Up, role=[importer-old], upNumber=21), Member(address = akka.tcp://ArchiveSystem@140.125.4.3:16666, Uid=807193699 status = Up, role=[web], upNumber=19)]. Node roles [importer,pubsub-node]
2017-12-09 16:03:36,013 [6] INFO  Akka.Cluster.ClusterCoreDaemon - Marking node(s) as REACHABLE [Member(address = akka.tcp://ArchiveSystem@140.125.4.2:16668, Uid=81451205 status = Up, role=[importer-old], upNumber=21)]. Node roles [importer,pubsub-node]
2017-12-09 16:03:36,013 [6] INFO  Archive.Shared.Cluster.ClusterStatus - UnreachableMember: Member(address = akka.tcp://ArchiveSystem@140.125.4.2:16668, Uid=81451205 status = Up, role=[importer-old], upNumber=21), Role(s): importer-old
2017-12-09 16:03:36,013 [6] INFO  Archive.Shared.Cluster.ClusterStatus - UnreachableMember: Member(address = akka.tcp://ArchiveSystem@140.125.4.3:16666, Uid=807193699 status = Up, role=[web], upNumber=19), Role(s): web
2017-12-09 16:03:36,013 [6] INFO  Akka.Cluster.ClusterCoreDaemon - Leader can currently not perform its duties, reachability status: [Reachability([akka.tcp://ArchiveSystem@140.125.4.1:16568 -> UniqueAddress: (akka.tcp://ArchiveSystem@140.125.4.2:16668, 81451205): Reachable [Reachable] (29)], [akka.tcp://ArchiveSystem@140.125.4.1:16568 -> UniqueAddress: (akka.tcp://ArchiveSystem@140.125.4.3:16666, 807193699): Unreachable [Unreachable] (28)])], member status: [$akka.tcp://ArchiveSystem@140.125.4.1:16568 $Up seen=$True, $akka.tcp://ArchiveSystem@140.125.4.1:16668 $Up seen=$False, $akka.tcp://ArchiveSystem@140.125.4.2:16568 $Up seen=$False, $akka.tcp://ArchiveSystem@140.125.4.2:16668 $Up seen=$False, $akka.tcp://ArchiveSystem@140.125.4.3:16666 $Up seen=$False, $akka.tcp://ArchiveSystem@140.125.4.4:16666 $Up seen=$False, $akka.tcp://ArchiveSystem@140.125.4.5:4053 $Up seen=$False, $akka.tcp://ArchiveSystem@140.125.4.6:4053 $Up seen=$False]
2017-12-09 16:03:36,138 [6] INFO  Archive.Shared.Cluster.ClusterStatus - ReachableMember: Member(address = akka.tcp://ArchiveSystem@140.125.4.2:16668, Uid=81451205 status = Up, role=[importer-old], upNumber=21), Role(s): importer-old
2017-12-09 16:03:36,138 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: 
2017-12-09 16:03:36,138 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,138 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,154 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,154 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,154 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,154 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,154 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,154 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address: akka.tcp://ArchiveSystem@140.125.4.3:16666, 
2017-12-09 16:03:36,170 [6] WARN  Archive.Shared.Cluster.ClusterStatus - Unreachable Member; Role: web, Status: Up, Address:
I have more than 2 roles ... "web" which is the front-end web application and "importer" which is the video processing service.
I noticed that after few days of running ... the combination of lighthouse and other roles ... somehow cause this
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 14:31
where are you hosting it?
jalchr
@jalchr
Dec 09 2017 14:31
This causes even internal (no remoting) message forwarding to not reach its destination
I'm hosting this on 6 VMs (2 web , 2 processing ... 2 lighthouse)
and for the record, the IIS recycles the web app after some idle time ... so it is natural to become 'unreachable'
I wish I can reproduce this
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 14:34
maybe try to add IIS hook to allow node to gracefully leave the cluster before going down?
unreachable is normal part when machine crashes or network connection goes down - if its possible its better to gracefully signal if node is going offline.
jalchr
@jalchr
Dec 09 2017 14:35
This is what I do in IIS ... asp.net app
//==============================================================
            HandleStarted();

            var properties = new AppProperties(app.Properties);
            var token = properties.OnAppDisposing;

            if (token != CancellationToken.None)
            {
                token.Register(() =>
                {
                    // do stuff here for ending / disposing
                    HandleStopped();
                    log.Info("ASP.NET application stopped !");
                });
            }

            log.Info("ASP.NET application started !");
Bartosz Sypytkowski
@Horusiath
Dec 09 2017 14:35
also there are several cases for non-graceful scenarios
jalchr
@jalchr
Dec 09 2017 14:36
private void HandleStopped()
        {
            log.Info("ASP.NET application stopped");
            if (ClusterSystem == null) return;
            log.Debug("Leaving cluster");
            ClusterHelper.Tell(new ClusterHelper.RemoveMember());
            var cluster = Akka.Cluster.Cluster.Get(ClusterSystem);
            cluster.RegisterOnMemberRemoved(() => MemberRemoved(ClusterSystem));
            cluster.Leave(cluster.SelfAddress);

            asTerminatedEvent.WaitOne(20000);
            ClusterSystem.Terminate().Wait(20000);
            log.Info("Actor system terminated, exiting");
        }

        private async void MemberRemoved(ActorSystem actorSystem)
        {
            log.Info("Member removed ...");
            await actorSystem.Terminate();
            this.asTerminatedEvent.Set();
        }
Internal messages fail:
2017-12-09 16:03:43,795 [56] INFO  Akka.Actor.LocalActorRef - Message Shutdown from akka://ArchiveSystem/system/sharding/FileHandler/67/%5C%5Cnas20%5Cd%24%5Carchive%5Cincoming%5Creuters%5C201712096110wd-northkorea-missilescontributors.xml/$c to akka://ArchiveSystem/system/sharding/FileHandler/67/%5C%5Cnas20%5Cd%24%5Carchive%5Cincoming%5Creuters%5C201712096110wd-northkorea-missilescontributors.xml/$c was not delivered. 4 dead letters encountered.
2017-12-09 16:03:43,826 [56] INFO  Akka.Actor.LocalActorRef - Message FileValidationResult from akka://ArchiveSystem/system/sharding/FileHandler/67/%5C%5Cnas20%5Cd%24%5Carchive%5Cincoming%5Creuters%5C201712096110wd-northkorea-missilescontributors.xml/$c to akka://ArchiveSystem/system/sharding/FileHandler/67/%5C%5Cnas20%5Cd%24%5Carchive%5Cincoming%5Creuters%5C201712096110wd-northkorea-missilescontributors.xml was not delivered. 5 dead letters encountered.