These are chat archives for damianh/Cedar

17th
Jul 2015
Dan Barua
@danbarua
Jul 17 2015 11:12
Getting the stream head on a stream with 262523 events is taking 600-700ms.. (yes yes yes I know i know i know I need to migrate it to CES before i get that crap out of my event store)
Damian Hickey
@damianh
Jul 17 2015 11:13
:)
Dan Barua
@danbarua
Jul 17 2015 11:13
Write 1000 times "An Event Store is not an Audit Log"
Might just have to suck it up and leave it running the migration over the weekend
"Limit  (cost=676.50..676.50 rows=1 width=13) (actual time=615.366..615.366 rows=1 loops=1)"
"  ->  Sort  (cost=676.50..676.63 rows=54 width=13) (actual time=615.362..615.362 rows=1 loops=1)"
or... i could optimize for ExpectedVersion == ExpectedVersion.Any and skip the current version check...
Dan Barua
@danbarua
Jul 17 2015 12:17
got that down to <50ms now, if you use ExpectedVersion.Any on a large stream then you're going to have to pay a penalty to find current_version.max, but if you're using ExpectedVersion.Any on a large stream you deserve to be punished for it I guess
Basically append with ExpectedVersion.Any is O(N) and ExpectedVersion.Something is O(1)
roughly
Damian Hickey
@damianh
Jul 17 2015 13:23
Yes, hmm
Think I'd like to optimise that
Should prob track the head version.
Dan Barua
@danbarua
Jul 17 2015 13:38
Possibly with a trigger on events, or as part of the insert function...
    SELECT streams.id_internal,
           streams.is_deleted,
           events.stream_version
    FROM streams
    LEFT JOIN events
          ON events.stream_id_internal = streams.id_internal
          AND (events.stream_version >= _expected_version OR _expected_version < 0)
    WHERE streams.id = _stream_id
    ORDER BY events.ordinal DESC
    LIMIT 1;
that's what I got
Damian Hickey
@damianh
Jul 17 2015 13:41
yep
João Bragança
@thefringeninja
Jul 17 2015 14:51
wait why do you need the version if you explicitly said you don't care
Dan Barua
@danbarua
Jul 17 2015 14:55
because CES sets the version
(i ported over the MSSQL impl)
we could do it in SQL somehow
no it doesn't
i'm being an idiot
basically we need the latest version to figure out what the next version is
if you have an ExpectedVersion, you can optimize this essentially by doing SELECT TOP 1 WHERE version >= expected_version
if you have no expected version, then the SELECT TOP 1 query could get very costly for a large append-only-don't-care-about-the-current-version stream
João Bragança
@thefringeninja
Jul 17 2015 15:00
ah claro, no autoincrement on a stream
Dan Barua
@danbarua
Jul 17 2015 15:00
NEventStore/NEventStore#393
because idiots, like me, do stuff like that ^
João Bragança
@thefringeninja
Jul 17 2015 15:00
course you could do one table per stream
Dan Barua
@danbarua
Jul 17 2015 15:00
that would make replays a bit unweidly
you could have a sequence per stream in dbs like oracle or postgres
Damian Hickey
@damianh
Jul 17 2015 15:00
one table per stream? lol
Dan Barua
@danbarua
Jul 17 2015 15:01
or we could use triggers on INSERT
sequence-per-stream looks good for me
Damian Hickey
@damianh
Jul 17 2015 15:05
beer o'clock!
Dan Barua
@danbarua
Jul 17 2015 15:05
CREATE TABLE events (stream_id_internal, stream_revision NOT NULL DEFAULT get_stream_sequence(stream_id_internal))
4pm here :(
i could use sequence-per-stream for stream version but there's no way to stop some idiot calling some_known_stream_sequence.next_val() and messing up the versioning...
Dan Barua
@danbarua
Jul 17 2015 15:54
danbarua/Cedar.EventStore@cea8bde
Dan Barua
@danbarua
Jul 17 2015 16:12
there might be dragons here, you could end up getting stream_version from uncommitted or aborted transcations
Important: To avoid blocking concurrent transactions that obtain numbers from the same sequence, a nextval operation is never rolled back; that is, once a value has been fetched it is considered used, even if the transaction that did the nextval later aborts. This means that aborted transactions might leave unused "holes" in the sequence of assigned values.
scratch that idea...