These are chat archives for damianh/Cedar

22nd
Jul 2015
Dennis Doomen
@dennisdoomen
Jul 22 2015 05:46
Then just calling response.EnsureStatusCode isn't going to sufficient. That'll throw HttpRequestException without any possibility to get hold of the actual HTTP status code. Any thoughts?
Damian Hickey
@damianh
Jul 22 2015 07:46
Do you mean response.EnsureSucessStatusCode() ?
If so, status code is in the exception message, but that's yuck. Don't use that, create your own exception.
Dennis Doomen
@dennisdoomen
Jul 22 2015 11:33
So how do you think Cedar should handle this, in case somebody wants to submit a PR?
Damian Hickey
@damianh
Jul 22 2015 12:57
Let me chew on this a wee bit
Dennis Doomen
@dennisdoomen
Jul 22 2015 13:39
:+1:
Dan Barua
@danbarua
Jul 22 2015 14:10
aha
blob
this is why my import keeps fucking up at this point. looks like clock drift.
Damian Hickey
@damianh
Jul 22 2015 14:11
ordering by datetime are we?
Order by commitstamp is fail
Dan Barua
@danbarua
Jul 22 2015 14:11
i'm not
Damian Hickey
@damianh
Jul 22 2015 14:11
oliver should have know better ffs
Dan Barua
@danbarua
Jul 22 2015 14:12
NES is on replay it looks like
well at least I haven't lost any events!
Damian Hickey
@damianh
Jul 22 2015 14:12
the stamp comes from the client
Dan Barua
@danbarua
Jul 22 2015 14:12
just out of order
yeah
Damian Hickey
@damianh
Jul 22 2015 14:12
it's a weak ordering
Dan Barua
@danbarua
Jul 22 2015 14:12
which is why we have the ordinal column in ces
Damian Hickey
@damianh
Jul 22 2015 14:12
there can be events with the same commitstamp in the same stream
yep
Dan Barua
@danbarua
Jul 22 2015 14:12
plus sql server truncates datetimes to 300ms anyway
Damian Hickey
@damianh
Jul 22 2015 14:13
time is not discrete
308000 events in that stream?
Dan Barua
@danbarua
Jul 22 2015 14:14
yes
totally does not belong here
hence moving to postgres so i can slice and dice and move it to somewhere more appropriate
Damian Hickey
@damianh
Jul 22 2015 14:14
ah
cool
Dan Barua
@danbarua
Jul 22 2015 14:15
it's an Append > ExpectedVersion.Any
i do have a use for it thoug
*though
at the moment we direct call log queries to a legacy system
for iterationNext we're going to pull from that system into our own call log
and i can use this stream to rebuild parts of that
Dan Barua
@danbarua
Jul 22 2015 14:20
right, quick patch for NEventStore v4 and we can crack on
Damian Hickey
@damianh
Jul 22 2015 14:22
ExpectedVersion.Any is good for streams of the non-domain event category
Dan Barua
@danbarua
Jul 22 2015 14:22
yeah - that's my 308000 events
in fact, >50% of the events i've shoved in NES are non-domain-events, just because it was convenient to do so
Damian Hickey
@damianh
Jul 22 2015 14:22
that is, no importance is attributed to it's position with respect to the previous & next event
yep
NES wasn't designed as a general event store
CES will be more suitable I think
Dan Barua
@danbarua
Jul 22 2015 14:23
yeah
Damian Hickey
@damianh
Jul 22 2015 14:23
But sql prob isn't the place for that stuff to be honest.
Dan Barua
@danbarua
Jul 22 2015 14:23
any thoughts on sharding/partitioning?
eg. schemas/buckets?
Damian Hickey
@damianh
Jul 22 2015 14:23
What are those events?
and what would happen if you lost them?
Dan Barua
@danbarua
Jul 22 2015 14:23
some are audit events eg. user played a recording
Damian Hickey
@damianh
Jul 22 2015 14:23
And what is the archival policy
Dan Barua
@danbarua
Jul 22 2015 14:23
keep forever
some are integration events to a backend system
eg. set the flag of this call to some value
will be used to migrate the call log into its own model
Damian Hickey
@damianh
Jul 22 2015 14:24
Do you do replays on them?
Dan Barua
@danbarua
Jul 22 2015 14:24
at the moment we drop redis and replay all the things
Damian Hickey
@damianh
Jul 22 2015 14:24
Partition around time is a reasonable strategy
i.e. stream per year
per day
per month
etc
Dan Barua
@danbarua
Jul 22 2015 14:25
i'd like to separate out the domain events
so drop projection views (a schema in postgres)
and rebuild from the domain events store
and then I don't have to sit through 800k non-domain events
just 500k domain events
Damian Hickey
@damianh
Jul 22 2015 14:25
For a start, an entirely separate store
db and/or server
Dan Barua
@danbarua
Jul 22 2015 14:26
for laziness/convenience i'd like to keep it in one db
but that said
Damian Hickey
@damianh
Jul 22 2015 14:26
You're not updating >1 stream in a transaction?
Dan Barua
@danbarua
Jul 22 2015 14:26
you don't NEED the audit log stream to debug/test
no
Damian Hickey
@damianh
Jul 22 2015 14:26
grand
Dan Barua
@danbarua
Jul 22 2015 14:26
never haha
Damian Hickey
@damianh
Jul 22 2015 14:26
;)
Dan Barua
@danbarua
Jul 22 2015 14:27
use process managers to cordinate workflows
that's next to look at
Damian Hickey
@damianh
Jul 22 2015 14:27
yar
Dan Barua
@danbarua
Jul 22 2015 14:27
i've got a domain events db and a saga events db
sagas being borrowed from CommonDomain
i don't think ES for sagas works well
This message was deleted
let's try that again
if (!this.CurrentReplayMessageTimeStamp.HasValue || this.CurrentReplayMessageTimeStamp.Value > new DateTime(2015, 03, 20, 12, 00, 00))
{
    // from 20/03/2015
    // all number allocations need approval from BSL accounts dept

    // we'll store the number so we can provision it later
    numbersToProvision[cmd.Number] = new InternalProvisionNumber(cmd.Number, cmd.DealerId, cmd.Lookups, cmd.Routing, cmd.Programme);

    // reserve the number
    Dispatch(new ReserveNumberForDealer(cmd.Number, cmd.DealerId, cmd.Routing));

    if (!dealerIsApproved)
    {
        if (!approvalRequested)
        {
            approvalRequested = true; // only raise this once.
            Dispatch(new RequestAccountsApproval(CorrelationId));
        }
    }

    // email Accounts dept so they can update the billing system
    Dispatch(new SendNumberAllocatedToDealerNotification(cmd.DealerId, cmd.Number, cmd.Routing));
}
else
{
    //prior to 20/03/2015 only outbound numbers needed approval

    if (cmd.Routing.Service != TelService.OutboundRecorder)
    {
        // inbound numbers can be allocated and provisioned without approval
        this.Dispatch(new InternalProvisionNumber(cmd.Number, cmd.DealerId, cmd.Lookups, cmd.Routing, cmd.Programme));
    }
    else
    {
        // outbound numbers need accounts approval
the logic changed on 20th march
which meant that we reacted to events occuring < 20th march with the new logic, causing commands to be dispatched
side effects being allocating phone numbers back to their previous owners
that was fun to debug
Damian Hickey
@damianh
Jul 22 2015 14:30
eek
Dan Barua
@danbarua
Jul 22 2015 14:30
So I think we'll move that to something using snapshots
I'll need to figure out how the whole commit/dispatch thing will work there