These are chat archives for akkadotnet/akka.net

25th
Sep 2017
jalchr
@jalchr
Sep 25 2017 07:19

Hi @Aaronontheweb @Horusiath ,
Regarding the message delivery in akka.net, I would love to see this at the library level.
I imagine one day Akka.net will have HOCON configuration like this:

#somewhere in remoting
message-delivery {
     type: "at-most-once", // default: extreme performance, possible message loss, no duplicates
    #type: "at-least-once", // no message lost, possible duplicates, great performance
    #type: "exactly-once", // no message lost, no duplicates, slowest performance
}

I wonder if we can mimic the "TCP/IP" acknowledgment model at the Actor level , so it happens "in-memory" as a persistence storage. Image instead of "IP packets" acknowledgment, we use the same model but for "actor-message" acknowledgment and guaranteed delivery.
So each message gets wrapped into an envelop that has meta data (ack, order, ... etc)
http://www.lovemytool.com/blog/2010/08/practical-tcp-series-sequence-and-acknowledgement-numbers-by-chris-greer.html

cyril andreichuk
@andreichuk
Sep 25 2017 09:45
hi all. are there any plans to add "large message sender/receiver" to Akka.Remote? it would make life much easier
ziya
@mtmk
Sep 25 2017 11:20
Hi guys. Quick question. Is it bad for actors to send themselves the poison pill? (Didn't want to use the 's' word ha ha)
Zetanova
@Zetanova
Sep 25 2017 11:41
@mtmk no, its not bad, after all the actor dont know the state of his mailbox. To push a sync/complete message to itself to process all following commands in order is fine. If possible its better to split operations in mutliple command-message sequences in gerneral
example would be Message.Disconnect { Stop = true }
the actor would process the "disconnect" and after if the Stop flag is set would push the Message.Stop after a Schduled intervall to itself
if the Message.Stop message is processed, there could be an Connect allready happend and the stop would be ignored
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 13:52

So each message gets wrapped into an envelop that has meta data

@jalchr not feasible:

  1. ACK-driven is one of many patterns. Others may be fire-and-forget, some may be batch-ack (or reactive streams), others may be NACK-driven. Fire and forget is the most atomic building block, every other pattern can be build on top of it. Adding ACK for every message essentially adds an extra cost (no matter if you want ACKs or not) and blocks you from using any other pattern.
  2. TCP works only in scope of a current connection session. It doesn't cover situations of peer resets etc.
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 13:58
think of mimicing UDP instead of TCP - you can build basically anything on top of it, including reliable protocols that are faster than original TCP (i.e. Aeron or QUIC)
Zetanova
@Zetanova
Sep 25 2017 14:03
@Horusiath If there is an AggregateRoot with ES, whats the best pattern to get some events included to generate some statistical aggregate values in the view? May current problem is that i am saving a reach-identity inside each event, that would be already redundant and the esjournal gets big fast
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:06
@Zetanova you can query over persistent event stream and build a materialized view (i.e. in form of SQL schema) from it
Zetanova
@Zetanova
Sep 25 2017 14:07
yes, that is what i am currently doing
even when i pre-aggregate the value like "network-bytes transmitted" the event gets because of the Identity (Guid + Name maybe parent-identity) big
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:11
@Zetanova isn't this a sort of problem on the serializer level?
Zetanova
@Zetanova
Sep 25 2017 14:12
yes, would need somehow a substitute for the event-serialization
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:13
what are you using for event serialization?
Zetanova
@Zetanova
Sep 25 2017 14:13
or a relable pattern to not persists or delete statistical events after aggregation
still waiting to switch, currently using json
even thinking about to create a child-actor with windows-event-log inside the AggregateRoot and pushing there the required events/messages to not perstists them
into the ES
Zetanova
@Zetanova
Sep 25 2017 14:19
and then work with the windows-performence native components to work with the data
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:19
this is probably a good idea, depending on the volume of your metrics - I know about patterns of working with heavy load metrics, but they are custom-made, there are no ready tools for them.
Zetanova
@Zetanova
Sep 25 2017 14:21
to relay on the windows-performanceCounter ?
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:21
no, using ring buffers for metric batching and custom binary encoding.
Zetanova
@Zetanova
Sep 25 2017 14:23
ok, thx
the "custom binary encoding" is implementable with Akka.Persistence ?
just to register a custom serializer with the required events
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:26
akka.persistence uses standard serializer interface - byte[] ToBytes(object o)/object FromBytes(byte[] bytes, Type t)
it's not the best thing out there, but it will work - however if you can observe problems on your side, maybe go into your own metrics provider and don't use akka.persistence
Zetanova
@Zetanova
Sep 25 2017 14:29
i think perfCounters with a custom file would be allready fine, maybe just to use newrelic or ApplicationInsights
to save the data is not that of a problem, but then to index it and pre-aggregate in different time periods is more work
somehow i am missing shared HA bit-resources (files) in self-hosted akka systems
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:34
afaik this is something up to your provider - most logs offer some sort of rolling files
Zetanova
@Zetanova
Sep 25 2017 14:37
lets say there is a stretched akka.cluster, for ES there is akka.persistence and its working, but file-resources there is nothing
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:38
what do you mean by "file-resources there is nothing"
Zetanova
@Zetanova
Sep 25 2017 14:39
cloud provider have there storage that can be synced
need somehow a file IO actor that can move or proxy the file over cluster-nodes
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:42
you could keep files locally and periodically read them updating a single global readonly view
Zetanova
@Zetanova
Sep 25 2017 14:45
thinking about a fileHandleActor that is globaly and staticly addressable, but can move/sync the file between DC's and something to provide the real-filehandle for an external component
if the file is stroed at DC1 and the filehandle would be aquired from DC2 it would be synced to DC2 and could then be accessed over path or a normal FileStream
dont know any good component that can do it in c# like rsync
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:51
what do you need filesync for?
Zetanova
@Zetanova
Sep 25 2017 14:52
because most components need a direct access to the file
only view have Stream support
if the file is localed somewhere on a node file-storage
a second node need to access it
UNC can work
but then again not all components work well with high latency file-access
simple example
there is an index-actor that holds a catalog of id-name paars and is working with a simple fast file-db
this actor is globaly addressable in the akka-cluster
and starts on a node as singleton
Zetanova
@Zetanova
Sep 25 2017 14:57
if it is starting it would need the last file-db from the old node
on his local filesystem
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 14:59
why file-db? You can easily put i.e. LMDB on every node and dispatch index across all nodes using consistent hashing / distributed hash table
Zetanova
@Zetanova
Sep 25 2017 14:59
its an example
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 15:00
or even easier (or harder) install something like cassandra, and let it take handle it all for you
Zetanova
@Zetanova
Sep 25 2017 15:00
ok an other one
the perfLog is stored localy in a binary file with the help of an non-akka component
the component requires an path to a local stored file
the actor that warps this component would require to be allways on the same cluster-node
or the file can be moved/synced with the actors location/site
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 15:04
this design assumes that perf logging works on a single node at the time
Zetanova
@Zetanova
Sep 25 2017 15:06
yes, many components do work with a file in file-lock mode
like some cli applications
Bartosz Sypytkowski
@Horusiath
Sep 25 2017 15:07
this will always be a bottleneck
Zetanova
@Zetanova
Sep 25 2017 15:10
will look into it, maybe it will work with the model of powershell-providers
something with "akka-filecluster-provider", "aws-s3-provider", "azure-bit-provider" and so on
my main thinking about it
is that the microservices should care about where the are hosted on the akka-cluster
chipdice
@chipdice
Sep 25 2017 19:54
@Horusiath - Do you know of any examples that would walk me through using protobuf for a specific message class I am sending? I'm not real familiar with protobuf, but believe it would compress these messages down to a size that will not exceed 32mb, as well as improve performance