Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Sep 18 08:45
    Zetanova commented #5152
  • Sep 18 07:37
    Zetanova synchronize #5152
  • Sep 18 07:15
    Zetanova commented #5273
  • Sep 18 05:12
    Zetanova synchronize #5273
  • Sep 18 04:51
    Zetanova synchronize #5273
  • Sep 17 16:56
    Aaronontheweb commented #5273
  • Sep 17 08:49
    Zetanova review_requested #5273
  • Sep 17 03:30
    Zetanova synchronize #5273
  • Sep 17 02:41
    Zetanova commented #5273
  • Sep 17 00:44
    Arkatufus closed #5284
  • Sep 16 21:14
    Arkatufus opened #5284
  • Sep 16 20:05
    Arkatufus synchronize #5282
  • Sep 16 19:47
    Aaronontheweb commented #5273
  • Sep 16 19:28
    Aaronontheweb commented #5273
  • Sep 16 19:18
    Aaronontheweb commented #5273
  • Sep 16 19:16
    Aaronontheweb commented #5273
  • Sep 16 19:15
    Aaronontheweb commented #5273
  • Sep 16 19:11
    Aaronontheweb commented #5273
  • Sep 16 16:52
    Aaronontheweb commented #4989
  • Sep 16 16:17
    dependabot[bot] labeled #5283
Sindre Alstad Wilting
@woksin
What happens when doing ActorOf on ActorSystem that ultimately resolved to the ActorPath of an already created actor? Does
Does it return the IActorRef of the already created actor?
David Mercer
@dbmercer
@woksin If the name already exists, you get an error.
Michael Handschuh
@mchandschuh

Consider a single stream of events. Each event belongs to a topic and has a sequence number. Sequences within a topic are guaranteed to be monotonic and continuous. I'm looking for a stream topology that allows subscribers to dynamically bind to topic specific sub-streams. If there's no subscribers then events can simply be dropped (Sink.Ignore<T>() works). Subscribers need to be able to gracefully disconnect from the topic sub-stream w/out affecting the rest of the running graph.

I think one way of doing this is to use GroupBy to produce sub-streams. I could then add a Broadcast to each sub-stream and by default add a Sink.Ignore<T>() to keep it drained. When a subscription for the topic comes in I could materialize a source from the Broadcast to dynamically add the consumer. For the purposes of this question, let's ignore issues caused by consumers processing messages at different rates as we already have a solution that addresses this. I think this runs into issues since GroupBy sub-streams never die IIRC. Ideally there would be a mechanism for retiring the processing pipeline for a particular topic that is guaranteed to never receive anymore events. All future consumers of the topic would simply replay from a persistent store.

My reluctance to use GroupBy is that it just doesn't seem like the right tool for the job. For some reason I feel like the PartitionHub is really what I want, but there's no way to track metadata on a per consumer basis that I'm aware of, or for the partitioner function to be notified of when consumers are added. I suppose a custom hub implementation could do the job, but that seems like overkill and would involve unnecessary implementation risks as well as implementing my own source materialized value to capture the consumer metadata (assuming I understand things correctly which is a BIG assumption!).

Another possible solution is to define an implementation of EventBus<TEvent, TClassifier, TSubscriber> that uses topic strings as the classifier. With this approach, publishing to subscribers is as simple as a Select(evt => bus.Publish(evt)). Inside the bus each TSubscriber could be a stream w/ a Source.Queue<TEvent> piping into a Broadcast and when someone subscribes to a topic, simply materialized the broadcast source which can then be used to feed the consumers processing pipeline. Leaning on the EventBus type here feels like I'm going against the grain of streams by not using a built-in stage for handling this fan-out to dynamic consumers. A plus side here is error isolation as it can't propagate through the event bus. We have a separate mechanism in place for handling replies/acks/confirms/processing errors/etc.

Thoughts?

Markus Schaber
@markusschaber

We're currently evaluating Akka.NET, and another team in our company therefore became aware of Akka.NET. They want to be able to "transparently" (whatever that exactly means) decide which code runs on the server and which code runs on the client (using client side blazor aka compiling .NET to WebAssembly). Security needs to be maintained as the clients are different users which potentially trust each other.

Is something like that generally thinkable using Akka.NET, maybe using Akka.Remoting with a custom protocol over websocket, or by using Akka.IO and "proxy actors" which forward and filter messages?

Sindre Alstad Wilting
@woksin
@Aaronontheweb Akka under the hood seems to spin up threads and run in the execution context of those threads when for instance creating actors. I have a problem where our system has an ExecutionContextManager with an AsyncLocal of our custom ExecutionContext object, I see that when the actor system actually gets to creating some of the actors I've told it to create the AsyncLocal ExecutionContext seems to be not in an expected state. Is it possible to hook in somewhere to make sure that I can set the value of this AsyncLocal to something before it gets to create the actors through the props? The reason I need this is because we have customized AutoFac to inject dependencies scoped as singletons per tenant and the tenant is derieved from our ExecutionContextManager's AsyncLocal ExecutionContext. I see in the debugger that when the actor systems tries to create some of the actors that needs tenant-bound dependencies that our ExecutionContext is never set. Could I do something with registering an IExtension to the ActorSystem? Or maybe I have to make a custom IActorRefProvider?
Sindre Alstad Wilting
@woksin
I just found Props.CreateBy and IIndirectActorProducer, that sounds like the thing I want. Does that sound right?
Manuel Islas
@trentcioran
Hi All,
I have a use case in which I need to process data from different sources, each needs to be expanded into its components, it is not a stream but more of a finite amount of work, composed of different stages of processing: text extraction or OCR if applicable, pre-processing, indexing for full-text-search, containerization of the extracted texts, transformation into HTML for display, etc. Items (files) expanded can change over time so I need to detect changed items and re-process them.
Something I don't see support for and that is the reason that I started to question the use of Akka.net is that I want to batch some of the expanded items, I can't process all at once as it could be on the tens of thousands per source, so it is not practical and better to work over multiple actors but would be best to batch this work so indexing is more efficient.
I have been reading the docs and build a prototype but feels like I might be over-complicating things, I would appreciate your feedback whether should I continue this path (akka.net) or should use something else.
I really appreciate your feedback and experience with Akka providing some light here. Thanks.
Aaron Stannard
@Aaronontheweb
@woksin sorry for the delay - was out last week on a brief paternity leave
I just found Props.CreateBy and IIndirectActorProducer, that sounds like the thing I want. Does that sound right?
@trentcioran that sounds like a great use case for Akka.Streams IMHO
a higher level abstraction on top of Akka.NET actors
which uses a LINQ-like syntax
to perform things like batching, debouncing, filtering, etc
Aaron Stannard
@Aaronontheweb
we have multiple users who use Akka.NET for this purpose BTW
I recall one in particular who built a large pre-processing system for managing massive Elastic indexing workloads
so you're in good company
Aaron Stannard
@Aaronontheweb
@markusschaber so running Akka.NET inside Blazor isn't really possible on the client side
due to WASM being single-threaded and restricting access to lots of primitives Akka.NET depends on (i.e. most synchronization mechanisms)
it can run fine in Blazor server mode though - done that here: https://github.com/petabridge/DrawTogether.NET
Is something like that generally thinkable using Akka.NET, maybe using Akka.Remoting with a custom protocol over websocket, or by using Akka.IO and "proxy actors" which forward and filter messages?
so this is what we do with things like https://cmd.petabridge.com/
it's all done via Akka.IO
with a strictly controlled serialization format to restrict access to what the client can do
we'll eventually introduce ACL and TLS to that
I think that approach is pretty feasible - the other approach would be to use something like Akka.gRPC (when we port it) to use gRPC as a secure gateway for invoking Akka.NET actors behind the scenes
Robbert
@Robbert-Driven-It
I hope someone here can point me in the right direction. I’ve been experimenting with clusters and the clusterclient. I’m able to communicate with a cluster from an external actor system however the clusterclient doesn’t seem to “fail over” when the cluster loses a node. My expectation was that the clusterclient would automatically keep track of new nodes (running a duplicate console app with a dynamic port) and would start sending messages there once the initial contact node has gone down… Am I mistaken, do I need to manually keep track of nodes?
Aaron Stannard
@Aaronontheweb
My expectation was that the clusterclient would automatically keep track of new nodes (running a duplicate console app with a dynamic port) and would start sending messages there once the initial contact node has gone down… Am I mistaken, do I need to manually keep track of nodes?
ClusterClient does automatically keep track of available nodes
however, if your client is on a node that terminates you will need to recreate it
grabbing some source
in the event that _clusterClient is talking to a node that is dead, the remote receptionist we were previously receiving messages from is now dead
so we just need to re-up our subscribe with a second ClusterClient.Send
in this case, I'm using a ReceiveTimeout to detect when messages coming from the remote client actor have stopped (since we're using pub sub)
a better way of doing it would be to Context.Watch the Sender for any of the messages I receive from the cluster
and then re-send the ClusterClient.Send after I receive a Terminated message
that will have me re-connect to another available node that has a ClusterClientReceptionist running
does that make sense?
@Robbert-Driven-It
Robbert
@Robbert-Driven-It
.. well kind of.. so you are saying the clusterclient keeps track of available nodes but if the cluster node my clusterclient is connected to terminates I need to instantiate a new clusterclient … why does is keep track of other receptionist nodes then? Only for the send all and publish scenarios? Feels odd
Robbert
@Robbert-Driven-It
I mean in order to keep track of available receptionist nodes it would need to monitor members up and down events..right? why wouldn’t it handle the scenario where the receptionist it’s currently connected to is going down and simply connect to one that’s available. I Have a feeling I’m missing something obvious:( or perhaps I’m not fully understanding the purpose of a clusterclient
Aaron Stannard
@Aaronontheweb

.. well kind of.. so you are saying the clusterclient keeps track of available nodes but if the cluster node my clusterclient is connected to terminates I need to instantiate a new clusterclient … why does is keep track of other receptionist nodes then? Only for the send all and publish scenarios? Feels odd

not odd

you have a tandem pair of actors
one on your client
one on the cluster node
if the cluster node shuts down, you just have to tell the client to connect to a new one via ClusterClient.Send
and that's only in a scenario where you're primarily reading from the cluster