These are chat archives for akkadotnet/akka.net

13th
Aug 2018
LiuGuowen
@LiuGuowen
Aug 13 2018 02:45
Hi Guys, recently I encountered a problem, there is a QA cluster with some kind of different services, I use a scheduler service to send "SyncOrder" command to another 2 order services, but finally we found a lot of messages are lost, it very strange, i don't know where is the root cause, do you have ideals about this ?
Nyola Mike
@nyolamike_gitlab
Aug 13 2018 04:23
Some one please help me, how do I coonect my akka.net system with signalr in asp.net core2.1
AndreSteenbergen
@AndreSteenbergen
Aug 13 2018 06:01
@nyolamike_gitlab Aaron made a github repository https://github.com/petabridge/akkadotnet-code-samples/tree/master/Cluster.WebCrawler In that crawler he made a docker system with SignalR. It would be too much info on gitter to explain all the steps. AkkaStartupTasks is called to provide the signalr actor the hub it needs. It is .net core 2.0, but it shouldn't be too hard to port to net core 2.1
Ismael Hamed
@ismaelhamed
Aug 13 2018 11:42
@Horusiath When tagging events in a PersistentFSM, are event adapters my only option? Being able to persist Tagged events directly, like in a normal PersistentActor, would be more convenient since I'm trying to dynamically assign tags for sharding purposes in the Read Side.
Bartosz Sypytkowski
@Horusiath
Aug 13 2018 11:43
@ismaelhamed the only thing that event adapter does is to map over your events before they are send to journal. You can do exactly the same right int your own code.
Bart de Boer
@boekabart
Aug 13 2018 15:22
@PinkyBrain_gitlab You have to introduce an interface IMyType<out T> where the out does the trick.
Question about Persistence. We're struggling with understanding how to recover from a database glitch during persisting (or recovery, for that matter). (in particular, SQL server db is unreachable during 'failover' )
We've learned so far that Persistent actors stop in such an event, which I can understand. What we've done already, is supervise them with a strategy that will recreate them OnStop, with a backoff strategy.
Should that be enough? Or is the persistent plugin itself also in a state that needs recovering?
Maybe @Horusiath could enlighten us a bit?
Bart de Boer
@boekabart
Aug 13 2018 15:28
It's not entirely clear what's going on in our application - but it is extremely SLOWLY processing data after such an event
Bart de Boer
@boekabart
Aug 13 2018 15:35
2018-08-13 15:27:40,049Z INFO [10 ] RepointableActorRef Message LoadSnapshot from akka://DivvVfdeDrdataTEST/user/xxxx/BackofSuperVisor/AssetDelta/keyed.7ef4129d-c7dd-4516-bffb-4b915f3e4e18 to akka://DivvVfdeDrdataTEST/system/akka.persistence.snapshot-store.sql-server was not delivered. 639 dead letters encountered.
is the clearest indication we have that SMTH is not right
so question is - how should an application handle an error during persist (or, recovery).
At this point in time, the only 'reliable' way we still see, is to kill the ActorSystem and restart the whole thing. That can't be right, right?
tiny hydra
@tinyhydra
Aug 13 2018 18:02
Hello. I'm fairly new to akka. I seem to have an issue where certain operations are taking exceptionally long to run. I'm running hundreds of actors, possibly over a thousand. My DB writes are timing out seemingly because the thread is repurposed and not returned in time. I'm speculating. Any thoughts?
tiny hydra
@tinyhydra
Aug 13 2018 19:06
Many of the actors are executing long-running WMI operations.
Bart de Boer
@boekabart
Aug 13 2018 19:34
Those (if not made Task-Async) will block your threads - hence limiting the # of parallel messages you can do to the # threads Akka gets/makes (from hocon config)
Try to limit the concurrency of the WMI operations to less than the # of worker threads
tiny hydra
@tinyhydra
Aug 13 2018 19:35
can I increase the worker threads?
Bart de Boer
@boekabart
Aug 13 2018 19:35
Yes, hocon. But you may want to consider the option above ^ for example, IIRC you can assign a dedicated thread pool to an actor/group of actors to prevent them from starving the general pool
tiny hydra
@tinyhydra
Aug 13 2018 19:36
the main issue here is that the operation is started, but appears to be immediately interrupted. If I could wait to start until I was sure I had the thread for some length of time...
Bart de Boer
@boekabart
Aug 13 2018 19:36
what do you mean with 'having the thread' ?
tiny hydra
@tinyhydra
Aug 13 2018 19:37
the operation starts, but times out. There's no reason for it to do that. The db is very responsive, but it appears something is stealing its thread and doesn't return it for >30s
providing a dedicated thread pool sounds worth a try. How would I go about that?
I don't mind funneling my operations through a single DB actor, or some other queue to limit concurrency for this case.
Bart de Boer
@boekabart
Aug 13 2018 20:02
Try the single Db actor (it's trivial to replace it later on with one that has N children to spread to load over)
tiny hydra
@tinyhydra
Aug 13 2018 20:16
I did that last week, it didn't seem to make a difference. I've implemented the pinned-dispatcher just now, which has a peculiar effect - The DB write still fails to complete, but the single actor is still receiving dozens of messages; it doesn't appear to be processing a single message at a time.
ah, it's not a single actor anymore for some reason. nevermind.
edwinparker
@edwinparker
Aug 13 2018 20:22
I'm trying to upgrade to .net 4.7.2 and having the same issue as posted here: akkadotnet/akka.net#3507 Is there any update on the issue?
tiny hydra
@tinyhydra
Aug 13 2018 20:50
Solved! Combination of changing the default dispatcher to have way more parallelism, and giving the db actor its own dispatcher. Thanks @boekabart !
Bart de Boer
@boekabart
Aug 13 2018 20:51
1 actor shouldn't have to have its own dispatcher (thread) though, normally, unless it's really time-critical
tiny hydra
@tinyhydra
Aug 13 2018 20:52
It's not so much the actor as the db client code, which is time sensitive due to an internal timeout.
I could increase the timeout, but it was still failing at 5 minutes. The combination of long-running WMI operations (ugh) and hundreds at a time seems to be a recipe for a dry thread pool.
Bart de Boer
@boekabart
Aug 13 2018 21:05
Ah, I thought you'd limit THOSE :)
Back to my own Persistence problem.
Plot Thickens a bit:
21:02:18,304Z WARN [48 ] HotFolderPersistentActor Rejected to persist event type [My.Actors.Io.HotFolderPersistentActor+ProcessedFileEvent] with sequence number [2] for persistenceId [HotFolderPersistent] due to [A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)].
It says : Rejected
but according to https://getakka.net/articles/persistence/event-sourcing.html , failures by 'transient' causes such as this ^^ network error, should be Failure
Main difference: Reject doesn't lead to actor Stop, Failure would
In my case, I need the Failure. Need the Restart.
@Horusiath would you not agree?
AndreSteenbergen
@AndreSteenbergen
Aug 13 2018 21:42
Would anyone be interested in a Identity4 Provider based on akka.net?