These are chat archives for akkadotnet/akka.net

16th
Feb 2016
ilhadad
@ilhadad
Feb 16 2016 00:09
I have created a persistence plug-in for CouchBase. I need some assistance on how to test it to make sure it plays nice. I based it off the MogoDB plug-in. I am not sure exactly how to use it. Perhaps if someone has some sample Mongo DB code I can look at then continue testing what I have created. By the way, I would like to publish this for the benefit of all.
Aaron Stannard
@Aaronontheweb
Feb 16 2016 00:10
@ilhadad have you run the Akka.Persistence.TestKit on top of it?
ilhadad
@ilhadad
Feb 16 2016 00:11
Well... not sure how to set that up either.
Aaron Stannard
@Aaronontheweb
Feb 16 2016 00:11
okey dokey
let me drop some links in here real fast
virtually all of the Akka.Persistence implementations use this exact build structure because it's what we standardize our projects on
makes it trivial for us to point our TeamCity build server at a new implementation and use it
you're going to want a copy of this for your own project - it's pretty much a copy-paste-n-edit
next up - the Akka.Persistence.TestKit
it's just a nuget package
designed to run a series of standardized tests that verify that your Akka.Persistence implementation matches the guarantees of the framework
to use it, you just need to author a journal and snapshot test harness like so
they're pretty much "fill in the blank"
the last thing you need to do, and I would love you forever if you can do this - although it's not possible with every database
if there's anyway you can, purely via scripting, download a local "ephemeral" instance of the database and run that from the commandline without needing anything to be pre-installed
I will love you forever because it means I get to avoid having to re-image our build servers
Aaron Stannard
@Aaronontheweb
Feb 16 2016 00:16
the Mongo plugin does this using Mongo2Go https://www.nuget.org/packages/Mongo2Go/0.1.4
and that's basically it
I can add CI support to the Couchbase repository easily enough - but I'm waiting on Azure to get back to me about transferring data over to a new Azure account (consolidating stuff)
so I might be delayed for a day or two while I wait on them
but after that we'll be good to go
ilhadad
@ilhadad
Feb 16 2016 00:21
I appreciate that. I will on getting it to pass the test. I may not be able to create an instance of the DB - let me investigate if CouchBase Mobile is able to do that. Alternatively, I can dump some sample data and export it for import in your own CouchBase install.
Aaron Stannard
@Aaronontheweb
Feb 16 2016 00:22
sure thing - whatever it takes to get it to work
just make sure you reply to my email about being willing to maintain it going forward :p
Aaron Stannard
@Aaronontheweb
Feb 16 2016 00:27
@boekabart so what exactly does your dispatcher do again? can you share a use case for it real fast just so I understand better?
joonhwan
@joonhwan
Feb 16 2016 05:25
@melcloud @christiansparre wow. it helped alot. appreciated.
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 08:45
@Aaronontheweb if I understood well, @boekabart dispatcher removes time factor from some of the assertions, replacing them by actual dispatcher occupation analysis
morganski
@morganski
Feb 16 2016 10:14
Hello - what's the pattern I should use for fork/join in Akka.Net? As an example I want to load a user profile which contains some user data (retrieved in Actor X) and some messages (retrieved in Actor Y). I send one message to the user supervisor (Z), which internally constructs X and Y, sends individual messages to each, waits for both responses to show up, and then replies to the original sender with a response that includes data from both X and Y. I can think of several ways to accomplish this but wondered what the conventional wisdom is on this? My current solution is to store the original sender in Z, it then Tells messages to X and Y and processes their responses (using Receive), tot these up and when both have arrived Tell the original sender. At the moment I have just two child actors, but I can see the logic in Z getting ever more complex the more child actors I need to add. Is there a better way to do this? Thanks in advance for your help.
Christian Sparre
@christiansparre
Feb 16 2016 10:20
Will the user supervisor deal with different "tasks" like one where A+B are involved, and one where B+C are involved? or will it always involve responses from all children?
alexgoeman
@alexgoeman
Feb 16 2016 10:26
Question related to serialization and persistence (+versioning). One of my concerns with using Persistence is to be able to deserialize all events (that ever existed) for duration of application lifetime. What is the right way to handle this ? There are three ways of versioning event strategies ,I know of. 1) Just create new classes (adding e.g. version suffixes) and keep supporting all classes 2) While deserializing, intercepting this and transforming into currently supported events 3)Migrating event data in the database to supported event types. Option 3I would like to avoid because of several reasons of maintenance. Option 1 requires that all types keep existing, Option 2 seems to be interesting, but looking at akka.net Serializers interface's derserialize method requires a Type object as a parameter, which also means that all types need to keep existing (?), so I am wandering how I can support refactorings of namespaces or refactoring of a classe's name ? All of these will result in type not being deserializeable anymore since Type will not exist anymore. Especially namespace refactorings are very difficult to avoid. What are the akka.net ways of intercepting the deserialization ? I see messages about not using Newtonsoft Json serializer and start using Wire, does wire have versioning options ? where can I find info about Wire ? I have got impression that some Akka.net features are required independent of the used serializer (e.g. how to redirect a class name string to another class name string (compatible in layout of course)) ?
morganski
@morganski
Feb 16 2016 10:30
@christiansparre - At the moment just the two, and (most likely), only ever responses from all children. So a literal Fork/Join, I'm not at present interested in partial responses.
Christian Sparre
@christiansparre
Feb 16 2016 10:31
@alexgoeman We are struggling with the same questions, we are not planning to use Akka.Persistence, but we have the same problems. For one we plan to keep out actual C# type information because it couples us hard to actual implemented types. We are also exploring rolling our own serialization with the features you mention in #2. It will probably take more effort and possibly rolling serialization for each event we have. But we will be in control :) We use MongoDB and will most likely still use BSON there as it is nice to actually be able to look at the events in the DB instead of some byte[] blob :)
Marc Piechura
@marcpiechura
Feb 16 2016 10:32
@christiansparre The MongoDB implementatiin of Akka.Persistence stores the events as readable documents ;)
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 10:33
@alexgoeman some of the persistent journals use their own, independent serializers for journals. However in other cases - when payload is serialized to database-unaware binary - if you can, you should not rely on somebody's else serializer and have your own - this way you can be sure that you'll always be able to get your data back, even when your system won't be using akka anymore. When it comes to serializers, both json.net and wire are version tolerant to some extend, but IMO schema based serializers (like Google Protocol Buffers, Flat Buffers or MS Bond) are the best for this purpose. Also about versioning strategy, 1 point (create new classes on each version) seems to be the best option. It's like serializing web api - because your old api/events may be continuously used, it's better to have them in the schema - you can utilize EventAdapters feature to upgrade old deserialized event's to current version of the fly.
I'm (very slowly) making an example, where I'm covering event versioning, saga patterns and cluster sharding scenarios.
Christian Sparre
@christiansparre
Feb 16 2016 10:49
@Silv3rcircl3 I know, we use it in a DDD like context and we need some sort of transactional behavior when persisting lets say 3 events, they either all get persisted or not, so we save them in a "batch" document. Akka.Persistence.MongoDb just does InsertMany. So for that usecase Akka.Persistence didn't quite fit our needs :)
alexgoeman
@alexgoeman
Feb 16 2016 10:50
@Horusiath , Having your own serializer , do you mean implemting your own serializer ? Is that not redoing something that has already been done by others, for me serialization in not a trivial task. So I would agree that you should be able to choose a serializer and it needs to be customizable and preferrable using some standards to not being dependent on some particular implementation. I agree for creating new classes for new events , but not sure what consequence you allude to with "because your old api/events may be continuously used" ? For me technique 2 (intercepting deserializing) would be used for cases like namespace or class name changes. Because for the app this is just one class but over the lifetime of app can have changed.
@Horusiath , but practically how can I support a renamed namespace or renamed class currently with akka.net serialization (I'm using Sql Server as journal store). Is there some akka.net interception point ? Because it looks to me that the fact that deserializer needs a Type that interception needs to be possible before calling the deserializer, e.g. some interception points in the journalstore. So is someone aware of such an extensibility point for the Sql Server store ?
Marc Piechura
@marcpiechura
Feb 16 2016 10:55
@christiansparre ok, then I haven't said anything ;)
Christian Sparre
@christiansparre
Feb 16 2016 10:57
I made a prototype "event store" implementation using Akka.net that sort of mimics the behavior of Akka.Persistence, but a little simpler. It's amazing how you can do a pretty complex thing very simple with akka :) I actually ended up having an actor named "Eventsourced" hehe
Zetanova
@Zetanova
Feb 16 2016 10:59
@alexgoeman currently the implementation is strict like @Horusiath himself :) If you used one Event in the persistence store, you are basicly not allowed to manipulate this event-message-class anymore. To create a new version of this event.message-class, you need to copy it to a other namespace or rename the copy. Of course you will need to handle this event-class in your actors like the first version. You will then have for both event versions a handler, even if maybe the first event-version-class will not used anymore.
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 11:05

@alexgoeman intercepting serialization on binary level (before deserializing data to particular object) won't help - i.e. if you have event E in version V1 with properties FirstName and LastName, but in version V2 this event E doesn't have these properties any longer, as they were replaced by a single FullName property. There is no built-in deserialization library able to cover such cases. You could try to upgrade V1 events to V2 directly in the database using some side app, but this is not possible in all cases - you had to close access to old events for the time of upgrade, and this cannot be done on the living system.

The best solution here is to deserialize all events to their original shape (V1 to V1, and V2 to V2), and then build a logic between journal and persistent actor, which will be able to translate V1 events to their V2 equivalent. Then you can send all events (now in V2 version) directly to persistent actor. And this something, that Event Adapters have been made for.

Christian Sparre
@christiansparre
Feb 16 2016 11:05
And now for something completely different: Is there a way in Akka.net to get a "visualization" of the actor tree?
alexgoeman
@alexgoeman
Feb 16 2016 11:10
@Horusiath : I completely agree about deserializing to the original shape. I only want to support renaming namespaces and classnames, because this will happen sooner or later in project lifecycle and they do not change the shape of a class.
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 11:12
@alexgoeman not easy to achieve, as from deserialization perspective 2 different full type names are 2 different objects.
same thing with changing serializer - it doesn't change shape of a class, but changes shape of it's binary representation
quite probable, that there are deserializers able to tolerate that switch, but personally I haven't tried that
Grzegorz Bernas
@profesor79
Feb 16 2016 11:37
@morganski could you post your question to stack overflow with akka.net tag? This will spread the knowledge out this chat (and SO is indexed by search engines). Thanks
morganski
@morganski
Feb 16 2016 11:39
@profesor79 - will do
Zetanova
@Zetanova
Feb 16 2016 11:48
@alexgoeman i looked at it some time ago, its currently not possible. Even to move the event-class to an other assembly, will break the deserializion and there is no substitution feature for it, what serializer have in some degree or form.
alexgoeman
@alexgoeman
Feb 16 2016 12:26
@Horusiath : so would it not be an idea to change the akka.net serializer interface by having the deserializer's method's object type defined as a string instead of Type parameter(type). Then a deserializer can be allowed the opportunity to decide to return another class type and the Type does not really have to exist in .NET assemblies.
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 12:28
@alexgoeman this was the idea behind changing PayloadType ⇒ Manifest column name. But it will not work in many scenarios (including using default serializers, which are emitting type name directly in payload itself)
alexgoeman
@alexgoeman
Feb 16 2016 12:31
@Horusiath : where do you have a PayloadType (my version of akka.net =>prototype= object FromBinary(byte[] bytes, Type type); I see the Manifest column indeed in the database. Do you mean the interface is being changed in next versions?
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 12:32
Manifest is used to identify serializer to handle the deserialization of a particular row among those registered in actor system
before akka v1.0.5 this column was named PayloadType
however I think, it's still not totally finished, when it comes to role I hope it will serve
alexgoeman
@alexgoeman
Feb 16 2016 12:35
In the type abstract class Serializer ? But the parameter type in .NET should be string and not of type Type to avoid having the type to exist
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 12:35
but I want to wait with this to upgrade Akka.Persistence to match the JVM version. If I remember correctly, after then, there will be some non-intrusive persistence schema changes
alexgoeman
@alexgoeman
Feb 16 2016 12:36
How is it defined in JVM version ? Not really sure what you mean with schema changes ?
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 12:37
when I was working on SQL-based plugins, in the meantime the new serializer type appeared SerializerWithStringManifest - however I think, these two are not yet cooperating nicely
eventjournal table schema
alexgoeman
@alexgoeman
Feb 16 2016 12:44
Schema changes seem to me not really intrusive, since that is encapsulated in the journal store. But changing the interface of a serializer seems to me much more intrusive. But can you agree that "object FromBinary(byte[] bytes, Type type);" should be changed to "object FromBinary(byte[] bytes, String type);", that this would allow for some extra flexibility ?
voltcode
@voltcode
Feb 16 2016 13:34
just reading the scrollback, gets really hard to comprehend if 2-3 discussions on questions are intertwined
Roger Johansson
@rogeralsing
Feb 16 2016 13:38
gitters scrollback is horrible, at least for me, it jumps all over the place..
Anthony Brown
@bruinbrown
Feb 16 2016 13:46
@alexgoeman It sounds like you might be looking for the SerializerWithStringManifest, it might be worth looking at, I'm not sure if it's been ported across yet http://doc.akka.io/japi/akka/2.4.1/akka/serialization/SerializerWithStringManifest.html
alexgoeman
@alexgoeman
Feb 16 2016 13:46
I'll check it out, thx
alexgoeman
@alexgoeman
Feb 16 2016 13:54
Indeed in java there is a string based method: java.lang.Object fromBinary(byte[] bytes, java.lang.String manifest)
From the docs : "For serialization of data that need to evolve over time the SerializerWithStringManifest is recommended instead of Serializer because the manifest (type hint) is a String instead of a Class. That means that the class can be moved/removed and the serializer can still deserialize old data by matching on the String. This is especially useful for Akka Persistence."
morganski
@morganski
Feb 16 2016 14:37
Question asked on Stack Overflow re: WaitAll with Akka - http://stackoverflow.com/questions/35434832/how-to-do-waitall-with-akka-net. Thanks in advance for any responses!
ilhadad
@ilhadad
Feb 16 2016 15:16
@Aaronontheweb With regards to the Akka.Persistence.Courchbase plugin, CouchBase does not have the concept of collections/tables, like MongoDB or SQL Server, respectively. Instead, all items are stored in a single bucket and differentiated by a document type field in the document. Do I need to still have Journal and Snapshot config? In theory, although not recommended, it is possible to have a bucket for snapshots and another for journal entries. The most likely scenario is that I can surface in the configuration files the ability to specify the document type value. The way I have it written now it all relies on one bucket and the document type values are "hard baked" into the code. Need some guidance here. My concern is more with the automated tests and sticking to the agreed expected functionality.
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 17:32
@ilhadad snapshot store and journal plugins can be provided separately, so your implementation doesn't necessarily have to cover both of them (programmer may choose to use i.e. couchbase journal and mongodb snapshots)
do you have some problems with satisfying testkit specs?
ilhadad
@ilhadad
Feb 16 2016 18:23
Ok got that. That's fine . I have separated them in anticipation of some strange reason I had not thought about. With regards to the testkit - not ready yet. I a few hours I will try.
Chris Constantin
@cconstantin
Feb 16 2016 19:47
Anyone know how this jvm akka code (Scala) translates to akka.net (destination is an ActorSelection)? ActorPath.fromString(destination.toSerializationFormat)
Not immediately obvious by looking at the scala and .NET code, at least not to me :)
I’m thinking this ActorPath.Parse(destination.PathString)
Bartosz Sypytkowski
@Horusiath
Feb 16 2016 20:13
@cconstantin you're right
Chris Constantin
@cconstantin
Feb 16 2016 20:13
great, thanks @Horusiath