These are chat archives for akkadotnet/akka.net

11th
Jun 2018
Ebere Abanonu
@eaba
Jun 11 2018 11:35
@Aaronontheweb I have a challenge for you! Are you in for it:
I need a demo for video/audio streaming using akka.net and https://github.com/hinaria/rtmp-sharp
It's challenge Monday, please kindly accept it; don't turn it down!
A guy did a server for it here:https://github.com/a1q123456/rtmp-sharp-server
Who is in for this?
Vagif Abilov
@object
Jun 11 2018 11:39
What would be the right way to investigate a problem related to timeout while replaying persistent events from an event journal in Microsoft SQL Server? We've been long suffering from timeout exceptions caused by persistent state not recovered within the specified time (60 seconds). We partly blamed ourselves because we were spawning large numbers of persistent actors with microstate instead small numbers with large state. However, errors shouldn't really be occurring as much as they are. We increased max size of connection pool from 100 (default) to 500 but it only buys us some time until exceptions come. It's all "Akka.Actor.AskTimeoutException: Timeout after 00:01:00 seconds" and they are all from persistent actors which have very small number of events each so they should not really keep connections open for more than a fraction of a second. We want to investigate this further but unfortunately SqlServer adapter doesn't log. Perhaps somebody who contributed to Akka persistence adapter can share some tips about what would be the best way to investigate such issue.
Vagif Abilov
@object
Jun 11 2018 14:53
I tried to profile it using SQL Profiler and I see that some INSERTs into EventJournal are not followed by calls sp_reset_connection. Can this be an indicator for a connection leak?
nathvi
@nathvi
Jun 11 2018 16:07
When looking at the speed of a local network connection, say, 10 GB /s. Does that mean upload + download GB/s?
Stijn Herreman
@stijnherreman
Jun 11 2018 16:11
@nathvi yes. Could be downloading at 10 GB/s, could be uploading at 10 GB/s, could be a mix of downloading and uploading at 10 GB/s.
Alex Hofer
@alexhofer
Jun 11 2018 16:35

Good Morning all,
I'm working on clustering for the first time and I am getting some errors when I kill and restart a node (via killing the node process and then restarting it). Below is how I am getting an actor ref at startup, which works fine, but when either node is removed and re-added to the cluster I get disassociation errors and the systems can no longer send messages to each other. Is there a better way to get those actor refs or send messages to a specific actor in a cluster from another node in that cluster?

It's important to note that I am communicating between two different projects, an API project and our main actor system project.

 var actorSelection = MyActorSystem.Current.ActorSelection(appSettingsActorConfig.GetActorUrl("MyActor"));
var systemResponse = actorSelection.Ask<ActorIdentity>(ident).Result;
SystemActors.MyActor = systemResponse.Subject;
Bartosz Sypytkowski
@Horusiath
Jun 11 2018 17:17
@alexhofer you always can use Tell over actor selection directly. It's a bit slower and won't verify if actor actually existed before trying to communicate with it (but this can be solved with ack messages). But actor selections don't care if destination actor will die and get ressurected later
Alex Hofer
@alexhofer
Jun 11 2018 17:18
Hmm, then is there another possible reason for disassociations when a node leaves and rejoins the cluster?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 17:18
the disassociation is normal
if the node is leaving gracefully you shouldn't see an error message
if the node leaves ungracefully and you end up with an unreachable node
then that could disrupt communication at startup
you can use https://cmd.petabridge.com/ to manually kick out unreachable nodes from the cluster if you want
but we also have some tools built-in to Akka.Cluster to help with that
but it sounds like you just really need a way of ensuring that the node leaves gracefully when you restart it
http://getakka.net/articles/actors/coordinated-shutdown.html - CoordinatedShutdown is the best tool for the job there
when you kill the process, let that Task it returns finish
that'll guarantee a clean exit
Alex Hofer
@alexhofer
Jun 11 2018 17:22
Ah ok, it looks like I misread (or more likely misunderstood) how clusters handle nodes that unexpectedly become unreachable. That helps, thanks! :)
Aaron Stannard
@Aaronontheweb
Jun 11 2018 17:23
no worries
I think I'm going to write a blog post on the subject of that
since I answer this question pretty frequently lol
Ebere Abanonu
@eaba
Jun 11 2018 17:54
@Aaronontheweb is it doable?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 17:55
is what doable?
Ebere Abanonu
@eaba
Jun 11 2018 17:55
@eaba
@Aaronontheweb I have a challenge for you! Are you in for it:
I need a demo for video/audio streaming using akka.net and https://github.com/hinaria/rtmp-sharp
It's challenge Monday, please kindly accept it; don't turn it down!
A guy did a server for it here:https://github.com/a1q123456/rtmp-sharp-server
Who is in for this?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 18:01
gonna have to take a pass on that lol - sounds like an interesting project but I'm already neck deep in other projects :p
Ebere Abanonu
@eaba
Jun 11 2018 18:02
how do we make you not to forget? Can we start with a repo for that?
Or can u offer a guide?
nathvi
@nathvi
Jun 11 2018 18:13
@stijnherreman , thanks
Alex Hofer
@alexhofer
Jun 11 2018 19:20

Ok @Aaronontheweb sorry I am still having issues, I enabled the split brain resolver and am trying out just the static-quorum, but I am still unable to communicate with a node after it leaves the cluster and comes back. I'm getting the following:

Resolve of path sequence [/user/myactorcoordinator#1845622221] failed

This is because when I store my actor refs to the core actor system TLAs in my web API project on startup they have their Uid's appended to their path in the actor identity result:

var selection = HubSystem.Current.ActorSelection(appSettingsActorConfig.GetActorUrl("MyActorCoordinator"));
var ident = new Identify(this.GetHashCode());
var response = selection.Ask<ActorIdentity>(ident).Result;

Results in:

[akka.tcp://actor-system@localhost:26100/user/myactorcoordinator#{Uid}]

So each time I restart the actor system it gets all new Uid's and my previous static references are now incorrect. So it's not my actor systems staying up thats the issue, its just my references to my actors becoming out of date. Should I not be using the result of that ActorIdentity ask and instead just use the ActorSelection that doesn't include the Uid?

Aaron Stannard
@Aaronontheweb
Jun 11 2018 19:21
ah yeah, don't include the UID in the actor selection
that's how we detect different incarnations of that actor
when processes restart
that UID will be random each time
if you deathwatch that actor reference
you should just re-run the process of reacquiring a new reference once the old incarnation is Terminated
and I'd do that with an ActorSelection that uses just the Address + path
without the UID
Alex Hofer
@alexhofer
Jun 11 2018 19:23

Ok that makes sense, this grabs the actor system with just the address and path:

HubSystem.Current.ActorSelection(appSettingsActorConfig.GetActorUrl("MyActorCoordinator"));

But then we proceeded to use the identity result instead, I believe just to make sure it was running and we got a response.

Aaron Stannard
@Aaronontheweb
Jun 11 2018 19:31
ah got it
I'd just re-run that original code when you detect a change in connectivity / availability of that actor
which deathwatch can do
or if you're subscribing to cluster events
can do it that way too
deathwatch is a bit more transparent / less code though
nathvi
@nathvi
Jun 11 2018 20:33
Does Akka support http transports?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:33
no
theoretically we could support HTTP/2 streaming
and we've looked at doing websocket transports before
for environments like Azure App Services where they lock down the ports
nathvi
@nathvi
Jun 11 2018 20:35
any plans on doing it in the future?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:35
not at this time
nathvi
@nathvi
Jun 11 2018 20:35
Too much work?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:35
other competing priorities
nathvi
@nathvi
Jun 11 2018 20:35
What's your top priority?
Kento
@robinsondotnet
Jun 11 2018 20:35
Is it possible to use "Ask" from a routerActor?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:36
@robinsondotnet the messages you ask the router actor will be forwarded to a routee
so the routee will complete the Ask
@nathvi Petabridge has some Akka.NET tooling that is coming out very soon for solving some pretty major DevOps pains when it comes to Akka.Cluster
so I'm personally spending a lot of time on that
nathvi
@nathvi
Jun 11 2018 20:37
ic
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:37
as far as Akka.NET itself goes though, we have a 1.4 release we're marching towards that has a bunch of interesting stuff
nathvi
@nathvi
Jun 11 2018 20:37
I might start working on a udp transport for akka
@Aaronontheweb , you have a link for everything in 1.4?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:37
multi-datacenter Akka.Cluster, StreamRefs, ValueTuple integration
Kento
@robinsondotnet
Jun 11 2018 20:37
@Aaronontheweb thank you,. what would be the best approach to achieve request-response and use router at the same time?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:37
sigh... no... I was supposed to publish a roadmap for that like two months ago
I need to get on that lol
(actively writing a different Akka.NET blog post right now)
nathvi
@nathvi
Jun 11 2018 20:38
sweeet
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:38
@robinsondotnet who needs to respond to the requesT?
one of the routees at the end of the router?
Kento
@robinsondotnet
Jun 11 2018 20:39
@Aaronontheweb Yes
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:39
Ask on the router should be fine
the router is transparent to the routees
so Ask functionality will continue to work
Kento
@robinsondotnet
Jun 11 2018 20:40
if I am using consistnt hashing routing strategy the response message has to implement their interfaces, right?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:40
ah no, just the request messaging
in terms of the IConsistentHashable stuff
the replies won't pass back through the router
they'll go straight to whoever sent the message to the router
in this case, that'd be one of the temporary actors Ask created
Kento
@robinsondotnet
Jun 11 2018 20:43
I see. thank you. I am sending messages directly to the router. Is it a bad practice?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:43
no
if I understand you correctly
oh wait
you mean your routees are replying back to the router itself ?
or are the routees replying to the Sender of the message?
Kento
@robinsondotnet
Jun 11 2018 20:44
thy are replying to the router itself
Aaron Stannard
@Aaronontheweb
Jun 11 2018 20:44
ohhhhhhhhh
yeah now I understand what you mean
no, that's not a good practice
the router can only send the message back to one of its routees
you should just reply to the sender
Sender.Tell etc
that'll send the message to whoever sent the original request to the router
Kento
@robinsondotnet
Jun 11 2018 20:45
ok. got it. thank you
Alex Hofer
@alexhofer
Jun 11 2018 21:20
Subscribing to Cluster events did it, I am getting all my refs refreshed! Thanks again @Aaronontheweb ! :)
Aaron Stannard
@Aaronontheweb
Jun 11 2018 23:03
I aim to please
Alex Hofer
@alexhofer
Jun 11 2018 23:10
Though, just for my own curiosity is there a way via something like ActorOf<T>() to just send a message to the cluster and have it go to an actor of a certain type? Since technically all of my actors belong to the same actor system on my cluster. Or is how I have it setup the way you normally do it?
Aaron Stannard
@Aaronontheweb
Jun 11 2018 23:10
clustered routers are one mechanism
does a lot of that for you
and cluster.sharding is the most heavy-duty option: https://petabridge.com/blog/introduction-to-cluster-sharding-akkadotnet/
in terms of what you just described
that probably most closely resembles cluster.sharding
Alex Hofer
@alexhofer
Jun 11 2018 23:13
Oh perfect, I will read over all that too! Thanks!