Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Dec 17 2022 04:45
    Nun022 opened #129
  • Dec 11 2022 21:14
    Build #1 passed
  • Dec 11 2022 21:11

    rofr on 0.8.0

    (compare)

  • Dec 10 2022 15:04
    Build #0 passed
  • Dec 10 2022 15:01

    rofr on master

    update testdependencies.sh update readme (compare)

  • Dec 10 2022 11:50
    Build #60 passed
  • Dec 10 2022 10:32

    dependabot[bot] on nuget

    (compare)

  • Dec 10 2022 10:32
    dependabot[bot] closed #128
  • Dec 10 2022 10:32
    dependabot[bot] edited #128
  • Dec 10 2022 10:32
    dependabot[bot] commented #128
  • Dec 10 2022 10:32
    dependabot[bot] edited #128
  • Dec 10 2022 10:31

    rofr on master

    upgrade to NET6, update nugets … (compare)

  • Dec 08 2022 13:49
    goblinfactory commented #128
  • Dec 08 2022 08:56
    CLAassistant commented #128
  • Dec 08 2022 08:56
    dependabot[bot] closed #127
  • Dec 08 2022 08:56

    dependabot[bot] on nuget

    (compare)

  • Dec 08 2022 08:56
    dependabot[bot] commented #127
  • Dec 08 2022 08:56
    dependabot[bot] labeled #128
  • Dec 08 2022 08:56
    dependabot[bot] opened #128
  • Dec 08 2022 08:56

    dependabot[bot] on nuget

    Bump Newtonsoft.Json from 12.0.… (compare)

Alan Hemmings
@goblinfactory
hahah... if it's got the wise RF chin scratching, not sure how much I can help ...
I'm quite good at seeing narrow opportunities to hack things, ...just when folk are saying, ha... we can't use this library it's rubbish, it will give a collission one in every million requests due to foo or baa or "clock drift" ... and I see it could be perfect for gaming ...
or banking. would be a big improvement on the current banking systems accurancy! lol
Robert Friberg
@rofr
the problem here is bigger, we could miss records while reading the tail due to commands arriving with an oldish timestamp from a funky node
Alan Hemmings
@goblinfactory
that makes sense
Robert Friberg
@rofr
that would put is in the mongodb category :)
Alan Hemmings
@goblinfactory

I've found this while we were chatting which is either marketing bullshit, or possibly something we could use?

Azure Storage simplifies cloud development
The Azure storage service supports all three strategies, although it is distinctive in its ability to provide full support for optimistic and pessimistic concurrency because it was designed to embrace a > strong consistency model which guarantees that when the Storage service commits a data insert or update operation all further accesses to that data will see the latest update. Storage platforms
that use an eventual consistency model have a lag between when a write is performed by one user and when the updated data can be seen by other users thus complicating development of client > applications in order to prevent inconsistencies from affecting end users.

is that possibly good news?
found here : https://docs.microsoft.com/en-us/azure/storage/common/storage-concurrency
Robert Friberg
@rofr
it won't help a late arriving record that get's inserted before other records because it has an older timestamp
Alan Hemmings
@goblinfactory
you mean problems like this ? cockroachdb/cockroach#13808
this sounds like ... if you can invent a protocol that could work around drift, you wouldnt have solved a big database problem, ...you'd have solved a big server clock problem, an even bigger market in terms of patent value?
Robert Friberg
@rofr
The problem remains even with perfectly synchronized clocks.
Alan Hemmings
@goblinfactory
if there was a work around for database folk, then the server clock manufacturers would use the same technique to improve their clocks.
Robert Friberg
@rofr
Memstate is not at all sensitive to the system time
Alan Hemmings
@goblinfactory
ok, I'll take the bait..."why does the problem still remain with perfectly syncrhonized clocks?" :D
Robert Friberg
@rofr
Latency.
Alan Hemmings
@goblinfactory
only if we're back to server generating the IDs
I may have misunderstood, ... your original suggestion that started this discussion was that we use reverse time ticks on clients, and add in a client ID?
Robert Friberg
@rofr
Well, if a node reads the tail and then a late arriving command is inserted then the read had a missing record
Alan Hemmings
@goblinfactory
so is that a non starter then? in order for it to be remotely possible, what building blocks are needed?
how have other tech offered what you'd like to see in memstate?
you have this already with cassandra right?
Robert Friberg
@rofr
Seems like a showstopper, yes. Anyway, I found an event sourcing implementation :)
Not cassandra
postgres, sqlstreamstore, eventstore are the main ones. Working on a provider for Pravega
Cosmos might work...
https://github.com/yevhen/Streamstone
Alan Hemmings
@goblinfactory
I have that on my trello board for domanium to investigate
cant' remember how I found it.
Robert Friberg
@rofr
and the answer is....
drum roll.....
Alan Hemmings
@goblinfactory
Ah, just read the homepage docs and remember where I saw it... I had googled table storage limitations, and it came up in top results for the limitations
Robert Friberg
@rofr
optimistic concurrency!
Alan Hemmings
@goblinfactory
dont you still need to sort clock problem?
Robert Friberg
@rofr
Let's just build it on top of Streamstone, that will be simple
Alan Hemmings
@goblinfactory
tbh ... I don't care how it's sorted, I know it's complicated, and solutions involve complicated stuff like gossip protocols, ...stuff I really don't want to care about.
streamstone looks great, from what I saw.
nice example here : https://github.com/OrleansContrib/Orleankka/blob/master/Samples/CSharp/EventSourcing/Persistence/Streamstone/Program.cs
ok... will throw something together tomorrow and compare that with single node, and see how they run...
assuming I don't ignore the compiler warnings ...
(groan!)
this will be fun :D
has been fun, except for... you know... the ...er... ignoring compiler warning thing.
Robert Friberg
@rofr
On top of StreamStone you won't need the BatchingJournalWriter, it already does batching
Alan Hemmings
@goblinfactory
Hi Rob, going to have to postpone eval of streamstone + memstate until this weekend. The big issue is not performance, but cost. Streamstone doesnt appear to support AzureTable storage, it only support Azure CosmosDb CloudTable which is similar but quite different cost wise.
I also probably won't be able to use it myself, since I need a solution that works with Azure functions, serverless style, that's pay as you go, with zero minimum monthly cost.
Alan Hemmings
@goblinfactory
Hi Robert,
so ...done a bit of experimenting (not a lot), and as far as I can see, stereamstone doesn't solve any problems I have. Originally I thought it would help solve the problem of ...what happens if azure decides to create a second instance of an azure function without me expecting it to, despite any attribute or json or configurations requiring a single partition or any other trick to try to force an azure function into some type of singleton
azure doesnt guarantee singleton instances, it gaurantees single instance will be processing a message at a time.
partitioning seems to be much more around azure functions that are not http triggered, but much more around message triggers