These are chat archives for exceptionless/Discuss

11th
Apr 2016
Blake Niemyjski
@niemyjski
Apr 11 2016 00:32
nope
This message was deleted
hmm that’s seems odd. let me know if installing the rewrite plugin fixes it
Blake Niemyjski
@niemyjski
Apr 11 2016 01:21
@srijken I’m going to test the dedup tomorrow and push out a release and finish up the server side stats
if you have any objections let me know :)
Sander Rijken
@srijken
Apr 11 2016 07:53
@niemyjski check
Blake Niemyjski
@niemyjski
Apr 11 2016 12:13
ok
Blake Niemyjski
@niemyjski
Apr 11 2016 12:44
how was everyones weekend?
Blake Niemyjski
@niemyjski
Apr 11 2016 13:25
@srijken @ejsmith
I noticed something with the .net client the other day and was wondering if you guys think we should fix it now before we release
if you take a look at the plugins.. the web api and mvc plugins run 100% of the time
and they set user info and do request info even if it’s already been set
so if you have an app that has both web api and mvc the web api stuff will run before mvc but will be taken out..
the issue with all of this is that we have a private info plugin that runs really early to set the user info to machine name..
then other things run and the web api plugin is towards the end that looks up the user info from the request
wondering if we need to make a service that is injected by mvc / web api for getting the user and falls back to environment.username
Blake Niemyjski
@niemyjski
Apr 11 2016 13:42
we’d need to define a interface and di helper to resolve the user identity
UserInfo GetUserIdentity(context)
IGetUserIdentity or something like that
what do you think
think it should fall back to environment.username if the interface doesn’t return anything?
Blake Niemyjski
@niemyjski
Apr 11 2016 14:24
I got it figured out :)
Blake Niemyjski
@niemyjski
Apr 11 2016 14:33
I think nuget is down
Blake Niemyjski
@niemyjski
Apr 11 2016 15:01
@ejsmith "I wanted to use your product in Xamarin solution(Android/iOS)... Which is provided via Pcl as far as I can understand... And it seemed to me like it's not storing unsent exceptions, so I looked at other solutions…"
with microsoft making xamarin free.. maybe it’s time we invest in a dedicated xamarin client that uses the pcl base..
Eric J. Smith
@ejsmith
Apr 11 2016 15:01
so talking about offline storage for pal?
pcl
Blake Niemyjski
@niemyjski
Apr 11 2016 15:02
I think so I asked for clarificiation
Eric J. Smith
@ejsmith
Apr 11 2016 15:02
I want to wait until RC2 comes out.
I think we can simplify things in the client then.
with the TFMs
Blake Niemyjski
@niemyjski
Apr 11 2016 15:03
yeah
TFM?
Matt Oswald
@moswald
Apr 11 2016 15:05
hey guys, I've been out of town for the past three days and didn't really get to read much on the PR I submitted
Blake Niemyjski
@niemyjski
Apr 11 2016 15:05
It’s all good :)
Matt Oswald
@moswald
Apr 11 2016 15:06
based on the feedback, I wonder if I should abandon it and let you guys discuss further, it seems like maybe you have larger plans
Blake Niemyjski
@niemyjski
Apr 11 2016 15:06
no pressure over here, we know you guys work on this in your free time and it’s greatly appreciated
Blake Niemyjski
@niemyjski
Apr 11 2016 15:45
Yeah, I think it was good to take a stab at it. I think there just needs to be some discussion around it.
We know we need to do it, just what’s the best way long term.
Eric J. Smith
@ejsmith
Apr 11 2016 15:46
@moswald yeah, we are just trying to figure out what the best approach is and you got the discussion started and I think that is awesome. I think we just need to iterate from there.
we just want to make sure that we come up with a solution that we all agree with.
Matt Oswald
@moswald
Apr 11 2016 15:47
roger that - I'm just here to help ;)
Blake Niemyjski
@niemyjski
Apr 11 2016 15:58
That’s greatly appreciated
I’m testing the .net client for release and then I’ll take a look at the pr and add some feedback
I saw that there was activity on it this weekend I just was busy with family
@srijken seems to be working really good :D
@ejsmith when I get this user stuff done
I’m adding the ability to view the raw event json
I need it like every day
Blake Niemyjski
@niemyjski
Apr 11 2016 16:05
Screen Shot 2016-04-11 at 11.04.35 AM.png
@srijken I’m taking a look into this
seems like when I click between two different errors multiple times I get a series of them
Eric J. Smith
@ejsmith
Apr 11 2016 16:08
yeah, I think that is definitely useful
and should be quick
Blake Niemyjski
@niemyjski
Apr 11 2016 16:15
yeah
Blake Niemyjski
@niemyjski
Apr 11 2016 16:24
@ejsmith @srijken
it appears we are doing
            hashCode = (hashCode * 397) ^ TotalPhysicalMemory.GetHashCode();
            hashCode = (hashCode * 397) ^ AvailablePhysicalMemory.GetHashCode();
            hashCode = (hashCode * 397) ^ ProcessMemorySize.GetHashCode();
            hashCode = (hashCode * 397) ^ (ThreadName == null ? 0 : ThreadName.GetHashCode());
            hashCode = (hashCode * 397) ^ (ThreadId == null ? 0 : ThreadId.GetHashCode());
which the memory one for process memory is changing by .1mb and causing it to not be deduplicated
wondering if thread id is changing it as well
Blake Niemyjski
@niemyjski
Apr 11 2016 16:44
hmm
total memory is fine
but I don’t think the memory and thread ones should be there
I’m seeing reference id interferring as it’s stored in a cookie
so it’s changing that on request info
should we remove cookies from the gethashcode logic too
?
Eric J. Smith
@ejsmith
Apr 11 2016 16:46
no
why would we do that?
if the cookies are different, don’t we want to see them?
Blake Niemyjski
@niemyjski
Apr 11 2016 16:46
because were updating the lastreferenceid cookie value and then it’s not deduped
yeah I guess but if everything else matches then it’s kind of a dup no?
Eric J. Smith
@ejsmith
Apr 11 2016 16:46
yeah, that sucks
if we can skip that one
that would be good.
Blake Niemyjski
@niemyjski
Apr 11 2016 16:47
only
yeah I’ll add logic to skip that key on gethashcode
you think we should remove the threadid,thread name, available memory and process memory size from gethashcode
to me those are things that could change alot
but having those may help track down multithreaded issues
Blake Niemyjski
@niemyjski
Apr 11 2016 16:58
DuplicateCheckerPlugin: Checking event: with hash: -1999577676
DuplicateCheckerPlugin: Checking event: with hash: -25846723
DuplicateCheckerPlugin: Checking event: with hash: -25846723
DuplicateCheckerPlugin: Checking event: with hash: -688792890
DuplicateCheckerPlugin: Checking event: with hash: -25846723
DuplicateCheckerPlugin: Checking event: with hash: -1999577676
DuplicateCheckerPlugin: Enqueueing event with hash:-1999577676 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-25846723 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-1999577676 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-25846723 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-688792890 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-25846723 to cache.
were using a concurrent queue
and it’s almost like it’s blocking
Eric J. Smith
@ejsmith
Apr 11 2016 17:11
I think removing those is reasonable
Blake Niemyjski
@niemyjski
Apr 11 2016 17:11
yeah I removed them now running into some kind of issue with the above
seems like there getting blocked when you queue up a bunch of them and then they are allowed
Eric J. Smith
@ejsmith
Apr 11 2016 17:12
don’t know what you mean
Blake Niemyjski
@niemyjski
Apr 11 2016 17:15
look at the above log message
Blake Niemyjski
@niemyjski
Apr 11 2016 17:21
updated the unit test and it’s failing
Blake Niemyjski
@niemyjski
Apr 11 2016 17:29
            c.UseLogger(new XunitExceptionlessLog(_writer));
:D
got awesome logging now in our client unit tests
Blake Niemyjski
@niemyjski
Apr 11 2016 17:37
Checking event: with hash: 738490750
Checking event: with hash: -717263284
Enqueueing event with hash:-717263284 to cache.
Checking event: with hash: -717263284
Enqueueing event with hash:-717263284 to cache.
Checking event: with hash: -717263284
Enqueueing event with hash:-717263284 to cache.
Enqueueing event with hash:738490750 to cache.
Checking event: with hash: -717263284
Checking event: with hash: -717263284
def having a threading issue
with the plugin
changed the forloop to a parallel 4 and everything broke loose
Blake Niemyjski
@niemyjski
Apr 11 2016 17:58
@srijken would you be willing to help me look into why the concurrent ones are failing? I added unit tests with improved logging
I think we might need to add some locking code in there
which is both scary and sucks
Blake Niemyjski
@niemyjski
Apr 11 2016 18:15
I Added a lock around all of it
and it increased the run time quite a bit
I’m also seeing a weird issue where 1 out of 4 runs fails with invalid hashcode
Enqueueing event with hash:-1799816918 to cache.
Adding event with hash:-1799816918 to cache.
Enqueueing event with hash:825815748 to cache.
Ignoring duplicate event with hash:-1799816918
Ignoring duplicate event with hash:-1799816918
Ignoring duplicate event with hash:-1799816918
Ignoring duplicate event with hash:-1799816918
Ignoring duplicate event with hash:-1799816918
Ignoring duplicate event with hash:-1799816918
Ignoring duplicate event with hash:-1799816918
same code paths and event being generated
Blake Niemyjski
@niemyjski
Apr 11 2016 18:46

DeduplicationBenchmarks_IdenticalExceptions, 116 runs

DeduplicationBenchmarks_IdenticalExceptions - 0.01ms

DeduplicationBenchmarks_LargeEventsFromFiles, 103 runs

DeduplicationBenchmarks_LargeEventsFromFiles - 0.30ms

DeduplicationBenchmarks_RandomExceptions, 110 runs

DeduplicationBenchmarks_RandomExceptions - 0.01ms

that’s what it was before my change
lets see what it is on the build server now
@ejsmith @srijken feedback on this is greatly appreciated: exceptionless/Exceptionless.Net@9a34941

DeduplicationBenchmarks_IdenticalExceptions, 114 runs

DeduplicationBenchmarks_IdenticalExceptions - 0.01ms

DeduplicationBenchmarks_LargeEventsFromFiles, 214 runs

DeduplicationBenchmarks_LargeEventsFromFiles - 0.31ms

DeduplicationBenchmarks_RandomExceptions, 112 runs

DeduplicationBenchmarks_RandomExceptions - 0.01ms

didn’t seem to slow it down any
Blake Niemyjski
@niemyjski
Apr 11 2016 19:09
@frankebersoll https://github.com/sergeyt/karma-typescript-preprocessor/issues/30#issuecomment-99655328 looks like the tooling is starting to support typescript a bit better :)
Sander Rijken
@srijken
Apr 11 2016 19:26
Hi, reading
agree on removing those properties from gethashcode
locking thing.. it’s weird that it’s necessary?
concurrent queue should fix concurrency issues
Sander Rijken
@srijken
Apr 11 2016 19:34

DuplicateCheckerPlugin: Checking event: with hash: -1999577676
DuplicateCheckerPlugin: Checking event: with hash: -25846723
DuplicateCheckerPlugin: Checking event: with hash: -25846723
DuplicateCheckerPlugin: Checking event: with hash: -688792890
DuplicateCheckerPlugin: Checking event: with hash: -25846723
DuplicateCheckerPlugin: Checking event: with hash: -1999577676
DuplicateCheckerPlugin: Enqueueing event with hash:-1999577676 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-25846723 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-1999577676 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-25846723 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-688792890 to cache.
DuplicateCheckerPlugin: Enqueueing event with hash:-25846723 to cache.

^^ What’s wrong with that?

except for the hash calculations
Blake Niemyjski
@niemyjski
Apr 11 2016 19:35
yeah
well those are all called concurrently
and they are all enqueued
when some of those should be ignored
Sander Rijken
@srijken
Apr 11 2016 19:35
that’s because the hashes don’t match up
Blake Niemyjski
@niemyjski
Apr 11 2016 19:36
yeah
look
Sander Rijken
@srijken
Apr 11 2016 19:36
so we must make sure they do match?
Blake Niemyjski
@niemyjski
Apr 11 2016 19:36
-2584673 all of those were enqueued and were the same number
same for the -1999577676
yeah there is something goofy going on
Sander Rijken
@srijken
Apr 11 2016 19:36
oh wait
it’s enqueueing them too often
Blake Niemyjski
@niemyjski
Apr 11 2016 19:37
yeah
idealy no lock would be good but if you look at it without one it’s chaos
Sander Rijken
@srijken
Apr 11 2016 19:38
can you add timestamps to the log output?
I’m pretty convinced it should work without the lock
the lock doesn’t fix the issue either right?
Blake Niemyjski
@niemyjski
Apr 11 2016 19:40
yeah we could do that by updating the xunit base class
xunitexceptionlesslog
it does part of the time
still fails randomly
Sander Rijken
@srijken
Apr 11 2016 19:40
yeah so there’s something else to it
Blake Niemyjski
@niemyjski
Apr 11 2016 19:40
yeah
nothing stood out to me
tried breaking it down
Sander Rijken
@srijken
Apr 11 2016 19:45
computer is slow
Eric J. Smith
@ejsmith
Apr 11 2016 19:46
more random test failures? :-(
Sander Rijken
@srijken
Apr 11 2016 19:48
@niemyjski what’s the exact testcase you’re running?
I think I foudn it :P
hm it works for me
Blake Niemyjski
@niemyjski
Apr 11 2016 19:54
run it multiple times and it will fail
@ejsmith just with dedup on multithreaded
@srijken I can reproduce it everytime by triggering new mvc errors
just click it the same error multiple times (api post and api get links)
Sander Rijken
@srijken
Apr 11 2016 19:57
the hash mismatch is caused in .Data
I noticed that .Data implements its own GetHashCode(), should use the extensions one
GetCollectionHashCode()
Blake Niemyjski
@niemyjski
Apr 11 2016 19:58
hmm
ok
weird
data should be exactly the same on all of those
nice catch
Sander Rijken
@srijken
Apr 11 2016 19:58
that one will order the keys before calculating the hashcode
it’s not what’s causing the problem though
Blake Niemyjski
@niemyjski
Apr 11 2016 19:58
hmm ok
Sander Rijken
@srijken
Apr 11 2016 19:59
but you can delete a bunch of code, which is always nice
Blake Niemyjski
@niemyjski
Apr 11 2016 19:59
ok
Blake Niemyjski
@niemyjski
Apr 11 2016 20:05
@srijken have you figured out a good way to see how gethashcode is being miscalculated
also, does each line need to be (dashcode ^ 397) or just the whole thing on the return?
Sander Rijken
@srijken
Apr 11 2016 20:07
it’s what R# generates. I read up on why it did that, and it sounded like a good idea
397 is a fairly large prime number
Blake Niemyjski
@niemyjski
Apr 11 2016 20:08
yeah
Sander Rijken
@srijken
Apr 11 2016 20:08
determining what causes it is a painful job
Blake Niemyjski
@niemyjski
Apr 11 2016 20:08
yeah :(
weird thing is it seems to happen on that unit test ever few executions
Sander Rijken
@srijken
Apr 11 2016 20:08
it’s coming from the Modules collection inside Error
Blake Niemyjski
@niemyjski
Apr 11 2016 20:08
hmm
wonder if it has to do with the module id calculation
cause that changes
maybe modules need to be sorted?
if so that’s a pita
do we even care about module info
@ejsmith
too me just because a module is added or removed an error occurs doesn’t mean anything even on module load errors
Sander Rijken
@srijken
Apr 11 2016 20:10
it’s not the order
Blake Niemyjski
@niemyjski
Apr 11 2016 20:10
we reassign the module id on every ToExceptionless()
are they different?
I compared them with beyond compare and they were all the same on my machine (checked 6 dupes that came through) only thing that was different was the reference id
Sander Rijken
@srijken
Apr 11 2016 20:12
checking
not the ID either
Blake Niemyjski
@niemyjski
Apr 11 2016 20:14
id, date and reference id stuff was different when I checked on the raw json serverside
Sander Rijken
@srijken
Apr 11 2016 20:16
what ID? event ID or ModuleID?
Blake Niemyjski
@niemyjski
Apr 11 2016 20:18
event id
Sander Rijken
@srijken
Apr 11 2016 20:20
the testcase itself isn’t stable either.. Right now I have the same consistent hashcode, and it fails because it gets 8 where it expects 9
that bit is a timing problem
when it gets 2 where it expects 1, it’s a hashcode problem
Eric J. Smith
@ejsmith
Apr 11 2016 20:21
I see my name was mentioned and there are like 100 messages?
Blake Niemyjski
@niemyjski
Apr 11 2016 20:22
yeah
so there are two different issues
@ejsmith yeah there were questions for you me and sander and working through them now
@ejsmith do you want me to merge in the EventStats into Foundatio Elasticsearch repository read only base?
Eric J. Smith
@ejsmith
Apr 11 2016 20:23
yeah, that would be nice.
Blake Niemyjski
@niemyjski
Apr 11 2016 20:23
or a seperate class?
Eric J. Smith
@ejsmith
Apr 11 2016 20:25
I think it would be good to have it as methods on the repo
Blake Niemyjski
@niemyjski
Apr 11 2016 20:25
ok
O
I’ll merge it in with models cause I need the terms stuff on our repo
Sander Rijken
@srijken
Apr 11 2016 20:30
ok, my search in .Modules was a stupid move. Conclusion now is that somehow the stacktrace is different
Blake Niemyjski
@niemyjski
Apr 11 2016 20:32
how
it’s the same stack trace :\
that’s what I don’t get
I get that it could be different in mvc due to maybe a race condition in handlers but crazy
@ejsmith there’s really no good way to do this in the base class:
.WithDateRange(utcStart, utcEnd, EventIndex.Fields.PersistentEvent.Date)
.WithIndices(utcStart, utcEnd, $"'{_eventIndex.VersionedName}-'yyyyMM");
Sander Rijken
@srijken
Apr 11 2016 20:37
hmf
getting quantum physics feelings right now
I’m inspecting where it’s probably failing
and it no longer fails
got it
Checking event:  with hash: 754157865 1407249123 GetException: 766225261 - ILOffset 8 - NativeOffset87
Checking event:  with hash: 752981631 1924808933 GetException: 217121131 - ILOffset 0 - NativeOffset87
Checking event:  with hash: 754157865 1407249123 GetException: 766225261 - ILOffset 8 - NativeOffset87
Checking event:  with hash: 754157865 1407249123 GetException: 766225261 - ILOffset 8 - NativeOffset87
Checking event:  with hash: 754157865 1407249123 GetException: 766225261 - ILOffset 8 - NativeOffset87
var error = context.Event.Data["@error"] as Exceptionless.Models.Data.Error;
            context.Log.FormattedInfo(typeof(DuplicateCheckerPlugin), String.Concat("Checking event: ", context.Event.Message, " with hash: ", hashCode + " " + error.StackTrace.GetCollectionHashCode() +
                 " " + string.Join(", ", error.StackTrace.Select(x => x.Name + ": " + x.GetHashCode() + " - ILOffset " + x.Data["ILOffset"] + " - NativeOffset" + x.Data["NativeOffset"]))));
so.. now what?
@niemyjski can you try this change, see if it fixes the real-world problem you were seeing?
Blake Niemyjski
@niemyjski
Apr 11 2016 20:42
yeah
Sander Rijken
@srijken
Apr 11 2016 20:43
hashCode = (hashCode * 397) ^ (Data == null ? 0 : Data.GetCollectionHashCode(new[] { "ILOffset" }));
Blake Niemyjski
@niemyjski
Apr 11 2016 20:43
wonder if we should get rid of native offset too
Sander Rijken
@srijken
Apr 11 2016 20:43
now my mac is slow
that code goes in Method
Blake Niemyjski
@niemyjski
Apr 11 2016 20:44
wonder why the iloffset would be different on the same executing code
Sander Rijken
@srijken
Apr 11 2016 20:44
maybe it’s some kind of optimization thing
JIT and pre-JIT or something
Blake Niemyjski
@niemyjski
Apr 11 2016 20:45
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\EnvironmentInfo.cs(131): hashCode = (hashCode 397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\InnerError.cs(64): hashCode = (hashCode
397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\Method.cs(42): hashCode = (hashCode 397) ^ (Data == null ? 0 : Data.GetCollectionHashCode(new[] { "ILOffset" }));
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\Module.cs(51): hashCode = (hashCode
397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\Parameter.cs(36): hashCode = (hashCode 397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\RequestInfo.cs(100): hashCode = (hashCode
397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\SimpleError.cs(53): hashCode = (hashCode 397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\UserDescription.cs(43): hashCode = (hashCode
397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Data\UserInfo.cs(52): hashCode = (hashCode 397) ^ (Data == null ? 0 : Data.GetHashCode());
C:\Code\Exceptionless.Net\Source\Shared\Models\Client\Event.cs(82): hashCode = (hashCode
397) ^ (Data == null ? 0 : Data.GetHashCode());
should we change all of those to GetCollectionHashCode?
Sander Rijken
@srijken
Apr 11 2016 20:45
yep
and dump DataDictionary.GetHashCode()
Blake Niemyjski
@niemyjski
Apr 11 2016 20:46
dump?
I changed all those to GetCollectionHashCode
Sander Rijken
@srijken
Apr 11 2016 20:47
the code inside DataDictionary, that can be removed
Blake Niemyjski
@niemyjski
Apr 11 2016 20:47
ok
Sander Rijken
@srijken
Apr 11 2016 20:47
“My understanding is that, when provided, the IL offset is simply mapped from the native offset using a table generated at JIT time (which may be an approximation due to optimizations). For the leaf native-frame, the native offset is where the exception was thrown while non-leaf native frames use the return address from the stack which may make it look like the exception was thrown by the instruction following the one where the exception actually happened."
Blake Niemyjski
@niemyjski
Apr 11 2016 20:49
hmm ok
so get rid of the equals and gethashcode overrides in the data
Sander Rijken
@srijken
Apr 11 2016 20:50
yeah
Blake Niemyjski
@niemyjski
Apr 11 2016 20:50
ok
wonderingi f we should leave it as it’s not hurting anything
this dedupe thing?
Blake Niemyjski
@niemyjski
Apr 11 2016 20:51
the equals and get hashcode on data dictionary
if you look on the data dictionary we are telling it to ignore tracelogs
Sander Rijken
@srijken
Apr 11 2016 20:52
hmf
complicated stuff
Blake Niemyjski
@niemyjski
Apr 11 2016 20:52
yeah
you changed it :D
lol
Sander Rijken
@srijken
Apr 11 2016 20:53
:)
so basically what we want is ignore @trace on Error.Data
Blake Niemyjski
@niemyjski
Apr 11 2016 20:55
yeah I guess
can add that to an exclusion I guess
and delete that equals code
seems to all be working great!
Sander Rijken
@srijken
Apr 11 2016 20:55
just pass that key to .GetCollectionHashCode()
Blake Niemyjski
@niemyjski
Apr 11 2016 20:55
that pull request you submitted couldn’t be merged
yeah
Sander Rijken
@srijken
Apr 11 2016 20:55
yeah I noticed, I’ll fix it
first fix this stuff :)
right now I’m checking why we need the locking
Blake Niemyjski
@niemyjski
Apr 11 2016 20:58
pushed the changes :)
they fixed all the issues with mvc!!!
glad I tested before deploying heehe
Sander Rijken
@srijken
Apr 11 2016 20:58
_processed.Any() is on IEnumerable, so that’s not threadsafe
Blake Niemyjski
@niemyjski
Apr 11 2016 20:58
hmm
yeah
Sander Rijken
@srijken
Apr 11 2016 21:00
we want queue behavior for cleanup
for the rest we want ConcurrentDictionary behavior
Blake Niemyjski
@niemyjski
Apr 11 2016 21:00
really wish I could step into these extension methods or go to source but all I see is empty definitions
yep
Sander Rijken
@srijken
Apr 11 2016 21:00
_mergedEvents.FirstOrDefault() has the same issue
Blake Niemyjski
@niemyjski
Apr 11 2016 21:01
yeah
Sander Rijken
@srijken
Apr 11 2016 21:01
hmm what to do
Blake Niemyjski
@niemyjski
Apr 11 2016 21:01
well I think the issue is that it’s not found in the queue and then other code runs then it enqueues it between checks and the lock prevents that
Sander Rijken
@srijken
Apr 11 2016 21:01
the lock makes the whole thing threadsafe yeah
btw, what’s the difference between FormattedInfo and FormattedTrace?
couldn’t get the Trace to show up in the test runner
Blake Niemyjski
@niemyjski
Apr 11 2016 21:05
not sure
oh
LogLevel
and there is a min level in the unit tests of Debug
could update that in GetClient
Sander Rijken
@srijken
Apr 11 2016 21:05
to get rid of the lock, we would have to keep both a concurrentdictionary (for checking if we have them) and a concurrentqueue (for cleanup of the dictionary)
Blake Niemyjski
@niemyjski
Apr 11 2016 21:05
or CreateClient on the plugin tests
well if we are locking do we even need concurrentqueue anymore?
Sander Rijken
@srijken
Apr 11 2016 21:06
no
Blake Niemyjski
@niemyjski
Apr 11 2016 21:06
and is concurrentdictionary even in the pcl?
Sander Rijken
@srijken
Apr 11 2016 21:07
we could also do a reader/writer lock
Blake Niemyjski
@niemyjski
Apr 11 2016 21:07
what ever you think is best and easiest to maintain :)
Sander Rijken
@srijken
Apr 11 2016 21:07
:)
it’s not that much harder
Blake Niemyjski
@niemyjski
Apr 11 2016 21:09
@ejsmith this stuff is pretty specific to exceptionless
hitting some roadblocks
Eric J. Smith
@ejsmith
Apr 11 2016 21:10
with the aggregations?
stuff like avg:somefield min:somefield, max:somefield
that shouldn’t be specific
Blake Niemyjski
@niemyjski
Apr 11 2016 21:10
i know
but we need access to the elastic index for alias name
Eric J. Smith
@ejsmith
Apr 11 2016 21:11
so what is specific?
Blake Niemyjski
@niemyjski
Apr 11 2016 21:11
can share my screen quick
Eric J. Smith
@ejsmith
Apr 11 2016 21:11
what do you mean alias name
can’t do that.
Blake Niemyjski
@niemyjski
Apr 11 2016 21:12
No generic way to inject the index on the context unless we add a constraint to the index with a <T> for the hell of it to auto inject it
then the date stuff is not generic but something we’d have to ahve
that’s only for terms tho
Screen Shot 2016-04-11 at 4.12.07 PM.png
            .WithDateRange(utcStart, utcEnd, EventIndex.Fields.PersistentEvent.Date)
            .WithIndices(utcStart, utcEnd, $"'{_eventIndex.VersionedName}-'yyyyMM”); 
is the biggest thing
you want to do smart date ranges
Sander Rijken
@srijken
Apr 11 2016 21:14
@niemyjski no ReaderWriterLock in PCL it seems, so just keep the lock the way it is?
Blake Niemyjski
@niemyjski
Apr 11 2016 21:14
yeah I guess
Eric J. Smith
@ejsmith
Apr 11 2016 21:14
that is up to the user to pass in a filter and system filter just like any other thing in the repo
aggregations are on top of that.
Blake Niemyjski
@niemyjski
Apr 11 2016 21:15
wonder if we can get away with non concurrent queues but is it worth it? what might it introduce
Eric J. Smith
@ejsmith
Apr 11 2016 21:15
use the exact same query builder stuff
Blake Niemyjski
@niemyjski
Apr 11 2016 21:15
yeah
ok
Sander Rijken
@srijken
Apr 11 2016 21:15
and, how much do you think we gain?
Blake Niemyjski
@niemyjski
Apr 11 2016 21:15
so object query as first param
yeah good idea :)
@srijken I’m not sure if anything but less chances to dead lock???
between there lock implementations and ours but I don’t see how that would happen with our locks being around everything
see first answer
that kinda suggests that it is in PCL?
oh where’s there’s a bunch of platforms
just leave it
could try a normal queue, and see if the tests + your recent problem are still OK?
Eric J. Smith
@ejsmith
Apr 11 2016 21:20
really hope the new TFM stuff will make this simpler.
Sander Rijken
@srijken
Apr 11 2016 21:21
TFM?
is that missing an R in front? ;)
Eric J. Smith
@ejsmith
Apr 11 2016 21:21
lol no
Target Framework Moniker
Sander Rijken
@srijken
Apr 11 2016 21:21
right
Eric J. Smith
@ejsmith
Apr 11 2016 21:22
the stuff where you target an API surface area and all platforms that support that contract or higher just wokr.
and you don’t have to do stuff for each platform
Sander Rijken
@srijken
Apr 11 2016 21:25
@niemyjski merge conflict is fixed
Blake Niemyjski
@niemyjski
Apr 11 2016 21:26
yay
feel free to merge it in
Sander Rijken
@srijken
Apr 11 2016 21:26
can’t ;)
Blake Niemyjski
@niemyjski
Apr 11 2016 21:26
I’m fine either way no?
Sander Rijken
@srijken
Apr 11 2016 21:27
ugh build error
Blake Niemyjski
@niemyjski
Apr 11 2016 21:27
yeah
Sander Rijken
@srijken
Apr 11 2016 21:28
it’s complaining about some reference. How can that be, I just changed one cs file
Blake Niemyjski
@niemyjski
Apr 11 2016 21:29
@srijken you have write access to that repo, do you not see any options to merge?
Sander Rijken
@srijken
Apr 11 2016 21:30
yeah I do
can you kick that build off once again to see if that does anything?
exceptionless/Exceptionless.Net@85ee49c
can’t see how that can break anything
Blake Niemyjski
@niemyjski
Apr 11 2016 21:31
yeah
kicking it off again
Sander Rijken
@srijken
Apr 11 2016 21:31
fails again
been getting those type references for a while
Sander Rijken
@srijken
Apr 11 2016 21:32
MSBuild version is now 14, last time it succeeded it was at 12
hmm
that’s MSBuild 12
and that’s like yesterday
Blake Niemyjski
@niemyjski
Apr 11 2016 21:33
I updated some dependencies to try and get benchmarks.net to run appears there is an issue with with there latest version
yeah what the hell
Sander Rijken
@srijken
Apr 11 2016 21:34
C:\Program Files (x86)\MSBuild\14.0\bin\Microsoft.Common.CurrentVersion.targets(1098,5): error MSB3644: The reference assemblies for framework ".NETFramework,Version=v4.5,Profile=Client" were not found. To resolve this, install t
he SDK or Targeting Pack for this framework version or retarget your application to a version of the framework for which you have the SDK or Targeting Pack installed. Note that assemblies will be resolved from the Global Assembly
Cache (GAC) and will be used in place of reference assemblies. Therefore your assembly may not be correctly targeted for the framework you intend.
but the build you just triggered when you did the hashcode changes on master is also MSBuild 14, and that one doesn’t have this error
Blake Niemyjski
@niemyjski
Apr 11 2016 21:36
yeah
I think I broke it
Sander Rijken
@srijken
Apr 11 2016 21:36
how? :)
Blake Niemyjski
@niemyjski
Apr 11 2016 21:37
idk
Sander Rijken
@srijken
Apr 11 2016 21:37
so.. now what?
Blake Niemyjski
@niemyjski
Apr 11 2016 21:37
unless appveyor changed things..
Sander Rijken
@srijken
Apr 11 2016 21:38
well, MSBuild doesn’t change version numbers all by itself
Sander Rijken
@srijken
Apr 11 2016 21:42
what happens when you kick master?
Blake Niemyjski
@niemyjski
Apr 11 2016 21:42
these are in master
Sander Rijken
@srijken
Apr 11 2016 21:43
“these”?
if you run it again, it should still be green
Blake Niemyjski
@niemyjski
Apr 11 2016 21:44
green or not
look at that commit and you can see in 43 that there are compiler related errors for system types
and that’s when the msbuild version changed :(
Sander Rijken
@srijken
Apr 11 2016 21:45
check
Blake Niemyjski
@niemyjski
Apr 11 2016 21:45
.Aggregations(agg => BuildAggregations(agg
.Terms("terms", t => BuildTermSort(t
.Field(term)
.Size(max)
.Aggregations(agg2 => BuildAggregations(agg2
.Min("first_occurrence", o => o.Field(ev => ev.Date))
.Max("last_occurrence", o => o.Field(ev => ev.Date)), fields)
), fields)
), fields)
Sander Rijken
@srijken
Apr 11 2016 21:45
it’s never easy is it?
Blake Niemyjski
@niemyjski
Apr 11 2016 21:45
not sure how we are going to do that
with numbers
nope :(
our numbers stats only support numbers and not dates
Eric J. Smith
@ejsmith
Apr 11 2016 21:46
should be able to do a min and max aggregation on the date.
min:first_occurrence max:last_occurrence
Blake Niemyjski
@niemyjski
Apr 11 2016 21:49
and where would we put those results
we stick everything in numbers
and those are dates
Eric J. Smith
@ejsmith
Apr 11 2016 21:50
what do you mean stick everything in numbers?
you mean the results are types as numbers?
Blake Niemyjski
@niemyjski
Apr 11 2016 21:50
yeah
Eric J. Smith
@ejsmith
Apr 11 2016 21:50
type them as objects
Blake Niemyjski
@niemyjski
Apr 11 2016 21:50
could change those to be objects I guess
and convert them on our end
Eric J. Smith
@ejsmith
Apr 11 2016 21:50
well the json will come down and it won’t matter
Blake Niemyjski
@niemyjski
Apr 11 2016 22:03
    public async Task<NumbersTimelineStatsResult> GetNumbersTimelineStatsAsync(object query, IEnumerable<FieldAggregation> fields, TimeSpan? displayTimeOffset = null, int desiredDataPoints = 100) {
        if (!displayTimeOffset.HasValue)
            displayTimeOffset = TimeSpan.Zero;

        // if no start date then figure out first event date
        if (!filter.DateRanges.First().UseStartDate)
            await UpdateFilterStartDateRangesAsync(filter, utcEnd).AnyContext();

        utcStart = filter.DateRanges.First().GetStartDate();
        utcEnd = filter.DateRanges.First().GetEndDate();
        var interval = GetInterval(utcStart, utcEnd, desiredDataPoints);

        var response = await Context.ElasticClient.SearchAsync<T>(CreateSearchDescriptor(query)
           .SearchType(SearchType.Count)
           .Aggregations(agg => BuildAggregations(agg
                .DateHistogram("timelime", t => t
                    .Field(ev => ev.Date)
                    .MinimumDocumentCount(0)
                    .Interval(interval.Item1)
                    .TimeZone(HoursAndMinutes(displayTimeOffset.Value))
                    .Aggregations(agg2 => BuildAggregations(agg2, fields))
                ), fields))
           .IgnoreUnavailable()
        ).AnyContext();
yeah that’s going to be fun...
to figure out what field to do the timeline on
guess I need to add a timeline aggregation with a few func’s eh?
side tracked on moving this over cause I need it for what I’m doing with counts per project on the repo
This message was deleted
Blake Niemyjski
@niemyjski
Apr 11 2016 22:08
@srijken
Sander Rijken
@srijken
Apr 11 2016 22:08
hm?
Blake Niemyjski
@niemyjski
Apr 11 2016 22:08
said they rolled out new vs2015 image
told me to use previous image
:(
unreal
Sander Rijken
@srijken
Apr 11 2016 22:09
I run VS2015 update 2
and it works
so..?
Blake Niemyjski
@niemyjski
Apr 11 2016 22:11
yeah
and we weren’t specifying a build image before
Sander Rijken
@srijken
Apr 11 2016 22:11
well I agree with running latest by default
otherwise a load of people never upgrade
Blake Niemyjski
@niemyjski
Apr 11 2016 22:11
yeah
I agree
there are some issues there that microsoft introduced
and until we drop .net 4.0 or move to standard lib I think they will be there
Sander Rijken
@srijken
Apr 11 2016 22:14
I’m off to bed
Blake Niemyjski
@niemyjski
Apr 11 2016 22:15
kk ty
Sander Rijken
@srijken
Apr 11 2016 22:15
the build thing, I’d switch back for now, wait what they come up with
and maybe have a branch that doesn’t have that image setting?
Blake Niemyjski
@niemyjski
Apr 11 2016 22:16
psuehd
pushed*
Sander Rijken
@srijken
Apr 11 2016 22:18
didn’t merge the PR yet, I’ll see tomorrow what happened ;)
Blake Niemyjski
@niemyjski
Apr 11 2016 22:19
ok
have a good night
Blake Niemyjski
@niemyjski
Apr 11 2016 22:49
        var range = (query as IDateRangeQuery)?.DateRanges?.FirstOrDefault();
        var interval = GetInterval(range?.GetStartDate() ?? DateTime.MinValue, range?.GetEndDate() ?? DateTime.UtcNow.AddHours(1), desiredDataPoints);
makes me wonder if that should just be a parameter
if you want fine grained intervals you call it
or specify a date range
might be really hard for us to figure out the right interval generically
        var range = (query as IDateRangeQuery)?.DateRanges?.FirstOrDefault();
        if (range == null || !range.UseDateRange)
            throw new ArgumentOutOfRangeException(nameof(query), "Query must contain a valid date range.");

        var interval = GetInterval(range.GetStartDate(), range.GetEndDate(), desiredDataPoints);
DateRange has a field property that we can use
but UseDateRange will return false if the start date is not greater than DateTime.MinValue so what if you want a range of all time?
wonder if well have issues with unlimited plans..
let me know if you want to look at this with me later