These are chat archives for akkadotnet/akka.net

9th
Aug 2015
Sean Gilliam
@sean-gilliam
Aug 09 2015 01:52
we usually branch per issue. but if you don't I don't think anyone would be too bent out of shape over it.
Ryan Davis
@rdavisau
Aug 09 2015 05:33
Yahoo, I managed to get a cluster up and running, @skotzko the docs are tight :+1: I know I can query the state of the nodes with the gossip methods, but can I perform an actor selection across the entire cluster?
Natan Vivo
@nvivo
Aug 09 2015 09:56
Does anyone here actually use Akka with eventsourcing in production?
Natan Vivo
@nvivo
Aug 09 2015 10:35
I'm starting to believe Akka.Persistence for EventSourcing is not something you'd do in practice. There are too many parts missing, like projections from multiple streams and projection recovery. And since that requires some additional support from the store and how events are stored, you cannot actually use any of the existing plugins. Did anyone have a different experience?
Marc Piechura
@marcpiechura
Aug 09 2015 10:44
@nvivo if I remember rightly, we have to wait until akka streams are implemented
Natan Vivo
@nvivo
Aug 09 2015 10:46
Right, I read something about that somewhere.
Looks like this will take some time even in scala, so the only solution is implement something from scratch today.
Roger Johansson
@rogeralsing
Aug 09 2015 10:54
Its the projections that will change with streams, persistent actors themselves will stay the same afaik
Natan Vivo
@nvivo
Aug 09 2015 10:54
@rogeralsing didn't you start a repo for streams?
Roger Johansson
@rogeralsing
Aug 09 2015 11:05
@Horusiath did
Natan Vivo
@nvivo
Aug 09 2015 11:07
I'll take a look. thanks
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 17:25
it's started (4,5k LOC now - I'm guessing that finally this will be like 6-9x more)
definitelly to much for one person working after hours
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 17:31
@nvivo what do you mean by projection recovery?
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:16
@Horusiath > Google.ProtocolBuffers dependency version missmatch
yeah, I had a bunch of trouble with the same thing for JSON.NET 7 when I was setting all of those Akka.Persistence builds up
the way I ended up solving it was to make sure that I only specified an upgrade for the core Akka packages
and let them dictate what versions of everything else needed to be installed
when I tried to upgrade a dependency independent of a package that also depends on it, that's when I got into trouble
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:18
after so many fuckups made by dynamic assembly loading checking this feels like a natural part of programming
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:18
I feel like a C++ programmer whenever I have to do that
I have an outstanding issue with the MultiNodeTestRunner I'm going to have to solve via AppDomain.OnResolve if that makes you feel any better :p
last time I had to do that was when I was doing C++/CLI programming
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:20
I like Haskell approach: if it compiles, it probably works - ofc that means, you first have to find your inner strength to make it compile
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:21
@nvivo the latest RC of JVM Akka includes a read-side query interface for Akka.Persistence
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:22
@Aaronontheweb link? I couldn't get true source code of this, their branching strategy is really weird
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:22
I think there's both an Akka.Streams interface and a low-level / direct-access-to-the-underlying-store one (unless I misunderstood @rkuhn)
@Horusiath I'm not sure yet myself
@nvivo I could have seen myself using Akka.Persistence in my MarkedUp example back in the day. It's a perfect fit for it. I essentially ended up having to build that into my app itself
basically at any given time I had millions of parallel state machines that had to journal their state transitions to Cassandra
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:26
I've done a little intro to akka.cluster for F# devs. Right now I want to provide a complete explanation of Akka.Persistence (it's almost ready) with all it's goods and limitations.
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:27
and the list of qualifying states that would result in a user receiving a message from us could change at any given time
the only downside of using Akka.Persistence would have been that if the current Akka.Persistence.Cassandra plugin doesn't support the ability to "expire" columns then I could end up with a lot of cold data taking up space in my C* cluster over time
even though each individual piece of data is small, when you're dealing with billions of events per month that adds up quickly
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:29
@Aaronontheweb in persistence, you may explicitly send message to journal to delete events permanently based on from-to sequence number criteria
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:30
@Horusiath yeah, but the persistent actors would only be alive for brief periods of time and a large number of them would go permanently inactive (when our customer's users would stop using the app)
I liked being able to handle that on the database-sied
side*
the big warning @skotzko and I have had to give to a lot of people interested in using Akka.Persistence lately has been that if you need the state actors are journaling to be readable by some BI or Reporting system outside your actor system
this is the wrong tool for the job
use a repository
it's new concept for most developers - this is state that belongs to your application, and your persistent store is just where the application goes to get it when the actors first start
the actor itself is still the single source of truth for whatever that state is
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:36
I really like the idea - the pattern that's dominated .NET for so long is "rush as fast as you can to ram this data into SQL Server and everything will be fine!"
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:37
yeah, sometime I hear that people want to use event sourcing backend as a data source for their BI - which is the first anti-pattern Greg Young mentioned when talking about event sourcing
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:37
@Horusiath yep, totally agree
I also like the idea of denormalization and using multiple databases for a given app
microservices kind of lends itself to the latter
we used SQL Server for all of our ACLs (access control lists) and billing events because that information was frequently read and needed to be ultra-consistent
but there wasn't very much of it - I could fit that entire database on an iPod touch even if we had 1000x more users
our analytics workload, however, would have imploded SQL Server
unless we had some business structure where cost was no object and I could just buy a ton of licenses and come up with some insane sharding strategy
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:42
sorry @Aaronontheweb, I'm in team Jacob
I mean, team Postgres :P
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:42
instead we picked a tool better suited for the job and would denormalize (and summarize) the information we needed for billing events back to SQL Server
@Horusiath I was going to say, I'm pretty sure Team Jacob wasn't one of the options from Twilight
... not.... not that I would know that...
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:43
:)
Natan Vivo
@nvivo
Aug 09 2015 18:44
@Horusiath I tried using your idea on akka cqrs and eventstream, but if I'm pushing data to a sql table for example, the app may crash or stop and some events may not be written to the table. I needed some form of recovery so I don't have to replay the entire stream when that happens.
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:45
@nvivo couldn't you just use a snapshots for that?
Natan Vivo
@nvivo
Aug 09 2015 18:46
How would you snapshot a sql table?
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:46
I mean actor state snapshots
persistence has build in snapshotting mechanism
Natan Vivo
@nvivo
Aug 09 2015 18:47
I thought about that, but i still would have to check if both were in sync, since I could snapshot before committing to the sal table or vice versa
*sql table
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:48
@nvivo are you using Akka.Persistence for this or a hand-rolled thing?
Natan Vivo
@nvivo
Aug 09 2015 18:48
But before that I bumped into the issue that persistent views couldn't read multiple streams
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:48
snapshots have persistenceId and sequence number of the last event applied at them moment, when snapshot has been made
Natan Vivo
@nvivo
Aug 09 2015 18:49
Without being able to read multiple streams, I can't do anything useful with ES
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:49
and persistent actor recovery uses them automatically
Aaron Stannard
@Aaronontheweb
Aug 09 2015 18:49
we passed 10,000 downloads of the core Akka package on NuGet this past week https://www.nuget.org/packages/Akka/ - didn't even notice
Natan Vivo
@nvivo
Aug 09 2015 18:50
@Horusiath this works fine for in memory data, but if the data is on a external store I didn't find a good way to keep both in sync
And by that, I don't mean the actor state stored on sql
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:50
@nvivo you still can read them, it's just not a build in. But in general - yes, this is a flaw of this design.
Natan Vivo
@nvivo
Aug 09 2015 18:50
I mean the data itaelf out of the actor
Yep, that's what I found - i have to build that
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:52
but I don't quite understand your sync argument - if I'm right you're saying that you're trying to use both Akka.Persistence for event sourcing and some custom commits to sql table?
Natan Vivo
@nvivo
Aug 09 2015 18:52
Another issue is about multiple streams and sequence numbers between streams. Greg Young's EventStore solves that, but I can't use that yet, so I ended up having to write thr akka plugin from scratch to handle a different store schema and mechanism
I need to create some projections in sql tables for integration and reporting
The story for persistence inside actors works fine
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 18:56
i, think that for this, you'll need a separate actor, which iterates over persistence backend and applies events onto sql projections. I feel that trying to do both in the same time (persisting event and updating a projection) is a recipe for disaster.
Natan Vivo
@nvivo
Aug 09 2015 18:57
Yep, that's my idea. I'm trying to see how much akka persistence works for that and how much I need to build myself. Because I see a lot of samples that Akka Persistence is good for event sourcing... But looks like that is only part of the story
So far it seems it works well for aggregates and nothing more
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 19:02
persistence story is still on the move - we'll have query side for it soon, and on the JVM there is also an alternative ES library for akka called eventuate - it addresses a lot of the problems, that emerged after akka.persistence came out
right now akka-persistence and eventuate are incompatible and hit different goals (akka-persistence is CP, eventuate is AP), but I hope that we could create some common ground for both
@nvivo if you want to experiment and will need some tech info, call me, I'll try to help
Natan Vivo
@nvivo
Aug 09 2015 19:08
Cool. Is there a repo for that? I'm out and on my phone right now, but when I get back I'll get in touch
it's described really well (something that akka-persistence forgot to do, when I started proting it :P )
Natan Vivo
@nvivo
Aug 09 2015 19:32
That's really interesting. I was thinking of creating something like that, probably much simpler of course, but tailored to my needa
Bartosz Sypytkowski
@Horusiath
Aug 09 2015 19:36
I'd really like to introduce some of it's ideas in akka-persistence. At least to the point when you could create a common persistence backend plugin and use it with both approaches.
Ryan Davis
@rdavisau
Aug 09 2015 20:52
@Horusiath, I saw your post on Akka.Cluster with F# - very timely for me as I had just set up the same kind of thing. I'm curious to know whether in your example you can send a DU over the cluster. Seems like I can send one to a local actorref, but to a valid remote selection (e.g. an actor somewhere else with a mailbox of printfn "%A" msg), strings and lists work, but a DU appears to come out the other end as seq [seq [seq []]] for a valueless token or seq [seq [seq []]; seq [seq [seq []]]] for something with a single value attached
Aaron Stannard
@Aaronontheweb
Aug 09 2015 20:57
@Horusiath IMHO, submit a PR to http://getakka.net/docs/clustering/cluster-overview with as an additional resource http://bartoszsypytkowski.com/how-create-an-akka-net-cluster-in-f/ - it's great IMHO
Ryan Davis
@rdavisau
Aug 09 2015 20:57
In my case the "remote nodes" and "local node" live in seperate projects, with the discriminated union defined in a shared project referenced by both. However for testing I have them all running in the same FSI session; to me this should rule out the possibility of either side not being aware of it
Aaron Stannard
@Aaronontheweb
Aug 09 2015 21:33
Akka.NET, F#, and RaspberryPi to build a robot arm
I love seeing what people are coming up with
jweimann
@jweimann
Aug 09 2015 22:40
nice
anyone here used akka.net with prism before btw? Haven't used prism much before so just looking for any guidance combining the 2 if anyone has some :)