Hi,
I have the following scenario:
1) 2 nodes clustered
2) Multiple singleton actors for folder watchers
3) Single api master in each node with routers (router = broadcast-group) that is aimed to distribute work across nodes
4) Multiple file handlers as Cluster-shards
Question is: how can I make the "Api Master" persistent and at the same time work on multiple nodes ?
I need to "remember" all files sent for processing.
Thanks,
@jalchr If I understand you right, you have one actor on each cluster node (API master) that you want to route messages through to reach/create folder watcher actors. And you want API master to know about the folder watchers on its node, so if the node restarts, they should spin up those folder watchers again? If so, then I've faced a similar headache. The way I solved it was to -not- make it persistent, and instead handle the "folder watchers" as cluster-shards and set config akka.cluster.sharding.remember-entities = on. That way Akka will remember them and restart them when they move nodes, or when nodes get added/removed and so on. You can have the folder watchers Tell the ApiMaster that they exist when they start/stop instead, so that each API master will know which folder watchers are running on that node -- instead of using persistence on the Api master. In other words, turn it backwards :)
So, instead of using a ReceivePersistentActor for your API master, and "recover" list of folder watchers, build it on the fly when the folder watchers are started/stopped.
NODE 1
| API master (Receive Actor) | Folderwatcher (Sharded entity) | |
|---|---|---|
| <- | Hello, I have been started! | |
| Add to list of folder watchers | ||
| <- | Goodbye, I've been stopped | |
| Remove from list of folder watchers |
You can have the API master forward the message to a cluster singleton if you want to keep track of all folder watchers in your entire cluster in a single place. Or you can use Fan/Collect to Ask all of the API masters in the cluster what folder watchers are there. Both should work.
Like so:
Folder watcher starts
Send Tell to local API master on same node
API master adds folder watcher to list of local folder watchers
Forward to cluster singleton
Cluster singleton adds folder watcher to global list of all folder watchers
If anyone else has a hat tric up their sleeve for solving this in some other way, that would be very interesting too.
I hope this helps (and that I understood your problem correct!)
public class FolderWatcher : ReceivePersistentActor
{
private readonly string folder;
public FolderWatcher(string folder)
{
this.folder = folder;
}
protected override void PreStart()
{
base.PreStart();
Context.ActorSelection("/user/api-master").Tell(folder);
}
}
public class APIMaster : ReceiveActor
{
private HashSet<IActorRef> localFolderWatchers = new HashSet<IActorRef>();
public APIMaster(IActorRef clusterSingleton)
{
Receive<string>(folder =>
{
this.localFolderWatchers.Add(Sender);
Context.Watch(Sender);
clusterSingleton.Tell(new FolderwatcherStarted(Sender, folder));
});
Receive<Terminated>(t =>
{
localFolderWatchers.Remove(t.ActorRef);
clusterSingleton.Tell(new FolderwatcherStopped(t.ActorRef));
});
}
}
public class FolderwatcherStarted
{
public FolderwatcherStarted(IActorRef folderWatcher, string folder)
{
this.Actor = folderWatcher;
this.Folder = folder;
}
public IActorRef Actor { get; private set; }
public string Folder { get; private set; }
}
public class FolderwatcherStopped
{
public FolderwatcherStopped(IActorRef folderWatcher)
{
this.Actor = folderWatcher;
}
public IActorRef Actor { get; private set; }
}
public class FolderwatcherDirectory : ReceiveActor
{
private readonly Dictionary<IActorRef, string> foldersWatched = new Dictionary<IActorRef, string>();
public FolderwatcherDirectory()
{
Receive<FolderwatcherStarted>(msg => foldersWatched[msg.Actor] = msg.Folder);
Receive<FolderwatcherStopped>(msg => foldersWatched.Remove(msg.Actor));
Receive<CheckIfFolderIsWatched>(msg => Sender.Tell(foldersWatched.ContainsValue(msg.Folder));
}
}
Hey @JessieWadman ... thanks for the extensive effort in bringing this out.
However, I thing we have a slight difference. Yes, I agree that ApiMaster can not be set to persist.
Rather, I'm thinking of creating a "RememberActor" which should be a singleton and persistent.
I'm planning to forward all messages to it from watchers.
Then forward them to ApiMaster, which distributes the load across nodes ...
Thoughts, anyone ?
IActorRef in the constructor. What's the proper way to stub this dependency and have it respond to certain messages, is it DelegateAutoPilot? I could also use some generic library like NSubstitute and stub Tell(), but it makes it a bit harder not having a Sender available.
