Hi guys.
Is there anyway to get "SequenceNr" while/after persisting a message?
Is there a way to delete a specific message with a specific "SequenceNr" ?
Is it necessary to use "SnapShots" with "Journals"?
What I'm trying to do is this:
I need to persist message for processing. Once each process finishes, I want to "clear" its message, so it don't get recovered on restarts ...
Any thoughts, with thanks
The problem with DeleteMessages is it assumes a sequence. But files might be processed out of order ... any solutions for such scenario ?
Another question: can we have a persistent singleton actor ? If yes, how to get an IActorRef for it ?
@jalchr
- You can get get the latest sequence nr using
LastSequenceNrproperty of persistent actor. - You cannot delete a single sequence nr in the middle of the event stream. This would corrupt the stream consistency.
- Snapshots are meant to be optimization mechanism for events. I think, that if you need simple state persistence it's better to implement is on your own.
About the rest: IMHO if you need to have persistent reliable message processing, just use message queue if you can. This can save your time and nerves. If you cannot use things like RabbitMQ, Azure Service Bus, Kafka or wathever else like that, you can use at-least-once-delivery actor, but I know that this is not a trivial thing to work with.
Regarding persistent singletons:
- If you have a very small number of singleton actors, you may want to use ClusterSingleton
- If you have more of them, the ClusterSharding is better option
Regarding the singletons, I was trying to use it with persistence, somehow it is NOT receiving messages. Once I turn it to a shard, it works.
Regarding the message bus, so you recommend a more reliable dedicated messaging system like Kafka or so. Is this going to work with remoting or replace it ?
Regarding “stream consistency” , if I follow you well, I think you are considering persistence as a pure “Event Stream”. In this case you are right about the sequence. What I was thinking about, is that persistence is some how more general, and that a sequenceNr is like a “record id”. Nonetheless, deleting a single message should be part of the api, IMHO.
@jalchr I don't know how you've configured your singletons:
- if you have heterogenic cluster, you probably should use role to scope cluster singleton to particular set of nodes.
- make sure you've started cluster singleton on each node capable to host target singleton, and cluster singleton proxy on each node that has not that capability but wants to communicate with it
- to communicate with cluster singleton use only
IActorRefs created by when using ClusterSingletonManager.Props / ClusterSingletonProxy.Props.
For the persistence part - akka.persistence is about eventsourcing, and as such the way for reverting an effect of an event is to emit a compentating event - this is i.e. how at-least-once-delivery works. AFAIK no single event sourcing provider allows you to just cut events from the middle without compromising the event stream safety.
For persistent queues - common pattern is to setup persistent queue in front of your actor system. Later on the message may be picked by actors and processed using standard akka communication patterns, then finally confirmed once processing has been done. If something goes wrong, you can simply pick it up one more time and try to process it again.