so ...done a bit of experimenting (not a lot), and as far as I can see, stereamstone doesn't solve any problems I have. Originally I thought it would help solve the problem of ...what happens if azure decides to create a second instance of an azure function without me expecting it to, despite any attribute or json or configurations requiring a single partition or any other trick to try to force an azure function into some type of singleton
private static Engine<AccountModel> _engine;
// static constructor for Azure function class Test1
static Test1()
{
// safety net in case not stopped.
AppDomain.CurrentDomain.ProcessExit += (s, e) => {
if(!stopped)
{
_engine.DisposeAsync().GetAwaiter().GetResult();
}
};
}
Hi @rofr. Recently I was looking into ways to hold a large singleton aggregate in memory but still have reasonable persistence to DB/file/etc. in order to recover state. I eventually got into event sourcing, Prevayler, and finally, memstate.
When I look at the github page, there appears to be infrequent development, and it's still in alpha/beta stage. Could you give some idea on how stable the current release is and what your intentions are for further development/support?
In addition, I have a question about using Postgres as the persistence. Suppose in the write-ahead approach, memstate attempts to persist a command C1 to Postgres. For whatever reason, the server persists C1, but we experience a client-side timeout. At this point, the in-memory state will be behind that of the persistence. If we then receive a command C2, that will persist to Postgres and then operate on the stale in-memory state and possibly result in a different state than when the commands are later replayed. Is there some mechanism in memstate to detect this kind of out-of-sync behavior (e.g. using event versioning) and recover transparently?
Hi @albertyfwu
I'll take the easy question first..
In addition, I have a question about using Postgres as the persistence. Suppose in the write-ahead approach, memstate attempts to persist a command C1 to Postgres. For whatever reason, the server persists C1, but we experience a client-side timeout. At this point, the in-memory state will be behind that of the persistence. If we then receive a command C2, that will persist to Postgres and then operate on the stale in-memory state and possibly result in a different state than when the commands are later replayed. Is there some mechanism in memstate to detect this kind of out-of-sync behavior (e.g. using event versioning) and recover transparently?
JournalRecords have a RecordNumber and by default these are required to have an unbroken sequence. In the scenario above, the engine will throw an exception when receiving C2 if it has not yet seen C1.
When I look at the github page, there appears to be infrequent development, and it's still in alpha/beta stage. Could you give some idea on how stable the current release is and what your intentions are for further development/support?
We are running memstate in production for several systems using EventStore or SqlStreamStore for storage. The core features are solid using either of these storage providers. There are some loose ends that need to be addressed before a 1.0 release though. Event subscriptions over a remote connection are not working for example.
PS: don't use the standalone Postgres provider, we may drop it altogether in favor of SqlStreamStore which has support for MySql, MSSQL and Postgres.
Hi @albertyfwu
I'll take the easy question first..In addition, I have a question about using Postgres as the persistence. Suppose in the write-ahead approach, memstate attempts to persist a command C1 to Postgres. For whatever reason, the server persists C1, but we experience a client-side timeout. At this point, the in-memory state will be behind that of the persistence. If we then receive a command C2, that will persist to Postgres and then operate on the stale in-memory state and possibly result in a different state than when the commands are later replayed. Is there some mechanism in memstate to detect this kind of out-of-sync behavior (e.g. using event versioning) and recover transparently?
JournalRecords have a RecordNumber and by default these are required to have an unbroken sequence. In the scenario above, the engine will throw an exception when receiving C2 if it has not yet seen C1.
Thanks for the quick response. I'm still unclear on when this discrepancy is resolved. After C1 is written to the DB (but unknown to the engine), how does the engine know to throw an exception when receiving C2? At that time, it wouldn't have learned that C1 was successfully written to DB yet. Is there some syncing that's happening out-of-band between C1 and C2? Or is there some synchronous synchronization happening to resolve discrepancies between engine in-memory and DB at the time it persists C2 to DB? Let me know if I need to explain this question more clearly.
In my particular use case, I have one single big aggregate (instead of many instances).
I know that memstate persists/applies commands sequentially, but I actually need the aggregate "locked" before the "commands" are issued, since I'm doing something more like "event" rather than "command" sourcing.
My workflow is:
- Lock the single big aggregate.
- Validate the command.
- If success, create an event, persist it, and apply it.
- Unlock the single big aggregate.
It seems that memstate supports something like this instead:
- Validate the request.
- If success, create a command.
- Lock the aggregate.
- Persist the command and apply it.
- Unlock the aggregate.
To me, it seems that I'd need to handle my own locking (before validation) before handing it off to memstate. Let me know if see how this can be more natively handled by memstate. Thanks.
After C1 is written to the DB (but unknown to the engine), how does the engine know to throw an exception when receiving C2?
Because C2 has the wrong RecordNumber. How it works is not trivial to explain as the various storage providers behave differently. An RDBMS backend could use either auto_increment or optimistic concurrency to ensure that commands are assigned sequential ids.
- You create a command object and pass it to
engine.Execute(cmd) - The engine sends the command to the underlying storage provider
- Storage provider writes the command
- The command as applied to the aggregate
- Validate your request with a memstate Query and issue a Command if the validation was successful
- Validate inside the Command.Execute method before making any changes to the aggregate. If invalid throw an exception or return a status code