Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Mar 06 12:55

    yevhen on master

    Fuck Russia! 🖕 (compare)

  • Feb 21 11:27

    yevhen on master

    Fix uploading (compare)

  • Feb 21 11:16

    yevhen on master

    Add nake script for bash Fix endpoint (compare)

  • Feb 21 11:07

    yevhen on master

    Fix yml (compare)

  • Feb 21 10:59

    yevhen on master

    Restore nake first (compare)

  • Feb 21 10:57

    yevhen on master

    Fix nake command (compare)

  • Feb 21 10:54

    yevhen on master

    Fix forking (compare)

  • Feb 21 10:46

    yevhen on master

    Fork azurite process (compare)

  • Feb 21 10:41

    yevhen on master

    Build azurite (compare)

  • Feb 21 10:40

    yevhen on master

    Init submodule (compare)

  • Feb 21 10:33

    yevhen on master

    Switch to Azurite (compare)

  • Feb 21 10:28

    yevhen on master

    Add azurite as submodule (compare)

  • Feb 21 09:42

    yevhen on master

    Update build script to Nake V3 (compare)

  • Feb 21 09:29

    yevhen on master

    Extend gitignore Upgrade to .net6 Update to latest Nake and VS im… (compare)

  • Mar 22 2021 17:29
    yevhen commented #61
  • Mar 19 2021 09:30
    iamim commented #61
  • Mar 19 2021 08:50
    yevhen commented #61
  • Mar 19 2021 08:50
    yevhen commented #61
  • Mar 18 2021 12:25
    iamim commented #61
  • Mar 15 2021 10:49
    yevhen commented #61
Yevhen Bobrov
@yevhen
got custom domain name back http://streamstone.net/ Thanks to the @ElanHasson for donation!!!
Matej Hertis
@mhertis
Special question, what can one do if Stream headers was manually changed in storage and now StreamStone is throwing `Streamstone.ConcurrencyConflictException at Streamstone.Stream+WriteOperation+Batch.HandleStream header has been changed or already exists in a storage :)
Yevhen Bobrov
@yevhen
Reload)
If stream is in memory the only way you can fix the situation is to reload it from storage to sync the etag
Matej Hertis
@mhertis
ok, thnx :)
was afraid of that answer :)
Robert Friberg
@rofr
I'm writing a storage backend for Memstate based on StreamStone, feels really solid so far. Nice work 👊
Yevhen Bobrov
@yevhen
@rofr thanks! If you need any help - don't hesitate to use this chat)
Robert Friberg
@rofr

@rofr thanks! If you need any help - don't hesitate to use this chat)

Thanks, good to know. I have a somewhat working version now:
https://github.com/DevrexLabs/memstate/tree/v2/src/Memstate.Azure

Robert Friberg
@rofr
Feel free to take a peek, any feedback would be greatly appreciated. BTW, what kind of throughput might I expect to see?
Yevhen Bobrov
@yevhen
@rofr ic what you're doing there) looks good
@rofr it's hard to predict throughput since it will depend on multiple factors, like batch size (volume) and how frequent are concurrent writes, as you don't have a single writer
Yevhen Bobrov
@yevhen
the hard cap for partition is 2K entities per second but it's for non-concurrent parallel writes. With event streams, you write sequentially due to optimistic concurrency (the writes are basically serialized) so it's different story
you might get 200-300 events per second (2-3 batches x100 written sequentially) if you're lucky with write latency and event arrival (ie you can form full batch very quickly).
Yevhen Bobrov
@yevhen
in case of concurrency when you write from multiple machines into the same stream - it will degrade considerably due to re-reads
so you can improve throughput by employing a single writer utilizing some sort of distributed application framework like Akka or Orleans. Actors are great hosts for streams
Orleans might give a boost to memstate by providing a way to distribute state - both sharding and replication, without sacrificing availability
Robert Friberg
@rofr
Thanks for the feedback @yevhen, I appreciate it
Robert Friberg
@rofr
I would classify memstate as a replicated state machine favoring consistency (CP) over availability (AP). Normally the entire system is a single (huge) event-sourced aggregate.
Robert Friberg
@rofr
OrigoDB, predecessor of memstate, has a sharding/partitioning but we haven't ported it yet as there is not much demand and there are other features with higher priority.
Robert Friberg
@rofr

so you can improve throughput by employing a single writer utilizing some sort of distributed application framework like Akka or Orleans. Actors are great hosts for streams

OrigoDB has a primary + read replica(s) architecture but no leader election, so failover requires manual intervention. But I have an idea about how to do leader election using commands embedded in the event stream allowing a primary + replica model for memstate as well.

Have you thought about supporting DynamoDB? I think the same schema and concepts can be applied there as well!
And how about cosmosdb?
cc @goblinfactory
Yevhen Bobrov
@yevhen
For CosmosDB check the Eveneum project from @jkonecki
I have no plans for DynamoDB at the moment
Yevhen Bobrov
@yevhen
pushed new release 2.3.1. We found some weird behavior from the underlying provider when dealing with large streams (more than 1 million). Figured out it was the non-optimal query. Presumably, after the fix, SS should now work much faster even with smaller streams. Try it out!
Matej Hertis
@mhertis
nice, will do ;)