Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Mar 22 17:29
    yevhen commented #61
  • Mar 19 09:30
    iamim commented #61
  • Mar 19 08:50
    yevhen commented #61
  • Mar 19 08:50
    yevhen commented #61
  • Mar 18 12:25
    iamim commented #61
  • Mar 15 10:49
    yevhen commented #61
  • Mar 15 10:46
    yevhen commented #61
  • Mar 15 10:46
    yevhen commented #61
  • Mar 15 10:25
    yevhen commented #61
  • Mar 09 12:25
    iamim commented #61
  • Mar 01 10:06
    iamim closed #63
  • Mar 01 10:06
    iamim commented #63
  • Mar 01 10:02
    yevhen commented #63
  • Mar 01 09:28
    iamim opened #63
  • Feb 24 15:01
    colgreen commented #54
  • Feb 24 11:17
    yevhen commented #54
  • Feb 23 19:28
    colgreen commented #54
  • Feb 16 15:26
    seniorquico commented #62
  • Feb 16 14:01
    yevhen closed #62
  • Feb 16 14:01
    yevhen commented #62
Elan Hasson
@ElanHasson
for a single stream, see S05_Read_from_stream
As far as creating projections for aggregate snapshots, I'm looking at https://liquidprojections.net to help automate that and make it easier.
You may also want to check out S06_Include_additional_entities, which includes a demo on how to include snapshots in the stream
Elan Hasson
@ElanHasson

Hi @yevhen, I'm persisting an event with a string Id to support dupe detection using my entity's natural key.

From SS perspective, How would I handle if the user wants to rename the property which backs the key?

Option 1: Keys are for life, do not support renaming of the natural key
Option 2: Update the row key for all objects in the stream (headache?)
Elan Hasson
@ElanHasson
Yevhen Bobrov
@yevhen
got custom domain name back http://streamstone.net/ Thanks to the @ElanHasson for donation!!!
Matej Hertis
@mhertis
Special question, what can one do if Stream headers was manually changed in storage and now StreamStone is throwing `Streamstone.ConcurrencyConflictException at Streamstone.Stream+WriteOperation+Batch.HandleStream header has been changed or already exists in a storage :)
Yevhen Bobrov
@yevhen
Reload)
If stream is in memory the only way you can fix the situation is to reload it from storage to sync the etag
Matej Hertis
@mhertis
ok, thnx :)
was afraid of that answer :)
Robert Friberg
@rofr
I'm writing a storage backend for Memstate based on StreamStone, feels really solid so far. Nice work 👊
Yevhen Bobrov
@yevhen
@rofr thanks! If you need any help - don't hesitate to use this chat)
Robert Friberg
@rofr

@rofr thanks! If you need any help - don't hesitate to use this chat)

Thanks, good to know. I have a somewhat working version now:
https://github.com/DevrexLabs/memstate/tree/v2/src/Memstate.Azure

Robert Friberg
@rofr
Feel free to take a peek, any feedback would be greatly appreciated. BTW, what kind of throughput might I expect to see?
Yevhen Bobrov
@yevhen
@rofr ic what you're doing there) looks good
@rofr it's hard to predict throughput since it will depend on multiple factors, like batch size (volume) and how frequent are concurrent writes, as you don't have a single writer
Yevhen Bobrov
@yevhen
the hard cap for partition is 2K entities per second but it's for non-concurrent parallel writes. With event streams, you write sequentially due to optimistic concurrency (the writes are basically serialized) so it's different story
you might get 200-300 events per second (2-3 batches x100 written sequentially) if you're lucky with write latency and event arrival (ie you can form full batch very quickly).
Yevhen Bobrov
@yevhen
in case of concurrency when you write from multiple machines into the same stream - it will degrade considerably due to re-reads
so you can improve throughput by employing a single writer utilizing some sort of distributed application framework like Akka or Orleans. Actors are great hosts for streams
Orleans might give a boost to memstate by providing a way to distribute state - both sharding and replication, without sacrificing availability
Robert Friberg
@rofr
Thanks for the feedback @yevhen, I appreciate it
Robert Friberg
@rofr
I would classify memstate as a replicated state machine favoring consistency (CP) over availability (AP). Normally the entire system is a single (huge) event-sourced aggregate.
Robert Friberg
@rofr
OrigoDB, predecessor of memstate, has a sharding/partitioning but we haven't ported it yet as there is not much demand and there are other features with higher priority.
Robert Friberg
@rofr

so you can improve throughput by employing a single writer utilizing some sort of distributed application framework like Akka or Orleans. Actors are great hosts for streams

OrigoDB has a primary + read replica(s) architecture but no leader election, so failover requires manual intervention. But I have an idea about how to do leader election using commands embedded in the event stream allowing a primary + replica model for memstate as well.

Have you thought about supporting DynamoDB? I think the same schema and concepts can be applied there as well!
And how about cosmosdb?
cc @goblinfactory
Yevhen Bobrov
@yevhen
For CosmosDB check the Eveneum project from @jkonecki
I have no plans for DynamoDB at the moment
Yevhen Bobrov
@yevhen
pushed new release 2.3.1. We found some weird behavior from the underlying provider when dealing with large streams (more than 1 million). Figured out it was the non-optimal query. Presumably, after the fix, SS should now work much faster even with smaller streams. Try it out!
Matej Hertis
@mhertis
nice, will do ;)