These are chat archives for exceptionless/Discuss

18th
Jan 2016
Chris Proud
@chrisproud
Jan 18 2016 15:00
I'll hold off that ES upgrade for a little longer then :smile: @niemyjski happy to help out with the testing when its ready.
Blake Niemyjski
@niemyjski
Jan 18 2016 15:00
yeah :)
I wish we were on it too. just a bit scary state right now. They broke a lot of things and the client is still alpha and we’ll have to reindex millions of events.
We should know this week what it’s all going to take :)
Eric J. Smith
@ejsmith
Jan 18 2016 15:11
I think we need to use this opportunity to think hard about switching to daily indexes.
Blake Niemyjski
@niemyjski
Jan 18 2016 15:19
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 16:00
so I added gitlink support to foundatio, but the appveyor.yml doesn’t run the gitlink command at the right time.
its doing all of our nuget packing for us, but I need gitlink to happen after the build and before nuget pack.
but that doesn’t seem to be possible using their stuff.
which means that we will have to do our own nuget packing.
Blake Niemyjski
@niemyjski
Jan 18 2016 16:01
:(
can we create a support ticket for it?
Eric J. Smith
@ejsmith
Jan 18 2016 16:01
I did.
no response yet.
looks like they have a few there.
guess that is what they are going to do with us?
Eric J. Smith
@ejsmith
Jan 18 2016 16:15
there are so many really cool things going on for the Docker world and tiny linux OSs
I can’t wait until we can run on linux and as a docker image.
Blake Niemyjski
@niemyjski
Jan 18 2016 16:42
yeah
would be nice
wish the docker for osx was a bit better
always have trouble setting it up
I think it’s because of the ubuntu requirement
or my limited knowledge :)
@megakid :)
Eric J. Smith
@ejsmith
Jan 18 2016 16:46
most likely you got niemyskified.
Blake Niemyjski
@niemyjski
Jan 18 2016 16:47
anyone know how I can sync my branches locally
so I do a branch, and then I delete them online and then sync locally in the github for windows client and it always has my old branches
like I want them to be removed
@srijken hows deduping coming along?
Blake Niemyjski
@niemyjski
Jan 18 2016 16:54
    /// <summary>
    /// 
    /// </summary>
    /// <param name="name"></param>
    /// <param name="id"></param>
    public EventBuilder SetEventReference(string name, string id) {
Anyone have something good to put in there to talk about relationships?
    /// <summary>
    /// Allows you to reference a parent event by it's reference_id property. This allows you to have parent and child relationships.
    /// </summary>
    /// <param name="name">Reference name</param>
    /// <param name="id">The reference id that points to a specific event</param>
Eric J. Smith
@ejsmith
Jan 18 2016 17:13
hrmm… Marvel is free for single node with ES 2.0
didn’t know that.
Blake Niemyjski
@niemyjski
Jan 18 2016 17:24
that’s pretty cool
Eric J. Smith
@ejsmith
Jan 18 2016 17:29
wonder how much longer they will let us renew our marvel license.
probably won’t work with 2.0
I don’t know if we will even be able to run those if we switch to 2.0 because I think they are really expensive for production use.
it’s always a bad sign when they won’t even show pricing info.
Blake Niemyjski
@niemyjski
Jan 18 2016 17:32
yeah
I can ask the elastic guys
I’ll make sure to to get it into the talk with them
lol
Eric J. Smith
@ejsmith
Jan 18 2016 17:33
ok
Blake Niemyjski
@niemyjski
Jan 18 2016 17:39
@all am I good or what?
Screen Shot 2016-01-18 at 11.38.35 AM.png
pushed on the branch :)
@all if you have any changes that need to make it into the .net client or js client releases please let me know ASAP otherwise it will be pushed on nuget later.
Eric J. Smith
@ejsmith
Jan 18 2016 17:54
Are those on the summary tab?
Blake Niemyjski
@niemyjski
Jan 18 2016 18:24
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 18:31
Cool
only thing I am worried about is if they have a bunch of them, but that would be pretty weird.
also, I still see session_id in the event json, we need to remove that.
Blake Niemyjski
@niemyjski
Jan 18 2016 18:36
elastic/elasticsearch-net#1734
already done
lol
Eric J. Smith
@ejsmith
Jan 18 2016 18:36
yeah, I ran into that the other day too.
yeah, I was thinking that we don’t need the dynamic template because it should already be a single token.
but maybe they have - or something in their ids
Blake Niemyjski
@niemyjski
Jan 18 2016 18:37
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 18:37
and that might cause it to get tokenized.
Blake Niemyjski
@niemyjski
Jan 18 2016 18:38
how much do we care about this
                .DynamicTemplates(dt => dt.Add(t => t.Name("idx_reference").Match("@ref:*").Mapping(m => m.Generic(f => f.Type("string").Index("not_analyzed")))))
I think that fixes it
Eric J. Smith
@ejsmith
Jan 18 2016 18:38
one thing to consider is that we might want to allow setting multiple reference ids.
the match is wrong
it will be in the idx field and it will be called blah-r
Blake Niemyjski
@niemyjski
Jan 18 2016 18:39
no idx will contain @ref:blah-s
Eric J. Smith
@ejsmith
Jan 18 2016 18:39
no it won't
Blake Niemyjski
@niemyjski
Jan 18 2016 18:39
yes it will
Eric J. Smith
@ejsmith
Jan 18 2016 18:39
no it wont
Blake Niemyjski
@niemyjski
Jan 18 2016 18:39
reread our discussions
it will because we don’t want it to conflict with existing values
Eric J. Smith
@ejsmith
Jan 18 2016 18:40
dude… that is the fucking custom property names
Blake Niemyjski
@niemyjski
Jan 18 2016 18:40
also we map it the right way
?
Eric J. Smith
@ejsmith
Jan 18 2016 18:41
go look at CopyDataToIndex
all idx fields have suffixes
it will be called session-r
Blake Niemyjski
@niemyjski
Jan 18 2016 18:41
did you add that?
I don’t remember coding that if I did
Eric J. Smith
@ejsmith
Jan 18 2016 18:42
and the -r is what will make it unique and also make it so that we can index it with different settings using dyamic templates.
yes, I added that.
the search processor will need to map ref.session -> idx.session-r
Blake Niemyjski
@niemyjski
Jan 18 2016 18:43
yeah just updated that
Eric J. Smith
@ejsmith
Jan 18 2016 18:45
need to change the dynamictemplate to be *-r
if that works.
Blake Niemyjski
@niemyjski
Jan 18 2016 18:47
    [HttpGet]
    [Route("~/api/v{version:int=1}/events")]
    [Route("~/api/v{version:int=1}/events/{authToken:token}”)
why is that v1
anything events is v2
we don’t ever do anything with auth token
James Connor
@megakid
Jan 18 2016 18:51
Is there anyway to add Username (Environment.UserName) to the list of Environment variables in an exception report? I am using DefaultData now
Blake Niemyjski
@niemyjski
Jan 18 2016 18:51
and we require auth access so they’d have to specify it as part of the query string
@megakid it will be automatically added as part of EnvironmentInfo
Eric J. Smith
@ejsmith
Jan 18 2016 18:52
yeah, guess I need to do more with that… wanted to test it some, but didn’t get to it.
I think I copied those routes from my other app.
Blake Niemyjski
@niemyjski
Jan 18 2016 18:52
gotcha
Frank Ebersoll
@frankebersoll
Jan 18 2016 18:53
hey there
Eric J. Smith
@ejsmith
Jan 18 2016 18:53
@megakid Environment.UserName should already be included with events.
no?
James Connor
@megakid
Jan 18 2016 18:53
hmm I dont seem to have it there. Just Machine Name and some other bits. I am using ExceptionlessWindowsExtensions.Register
Frank Ebersoll
@frankebersoll
Jan 18 2016 18:53
i won’t get persistance into the js client for this release, but for the next.
Eric J. Smith
@ejsmith
Jan 18 2016 18:53
hrmm.. guess we just have machine name.
James Connor
@megakid
Jan 18 2016 18:54
c.Configuration.DefaultData.Add("UserName", Environment.UserName);
I am using that, seems to work ok
Eric J. Smith
@ejsmith
Jan 18 2016 18:54
I think we do use Environment.UserName for setting a default user ideneity for somethings though.
is Environment.UserName the person that is running your app?
James Connor
@megakid
Jan 18 2016 18:54
yes, we use NT login for our applications
Eric J. Smith
@ejsmith
Jan 18 2016 18:54
if so, you should call SetUserIdentity(Environment.UserName)
James Connor
@megakid
Jan 18 2016 18:54
ahh
Eric J. Smith
@ejsmith
Jan 18 2016 18:55
and I am pretty sure we have code to do that some place.
James Connor
@megakid
Jan 18 2016 18:55
PrivateInformationPlugin
seems strange it doesn't add to my reports
unless I have to add that plugin manually
Eric J. Smith
@ejsmith
Jan 18 2016 18:56
It’s in the windowsextensions and I think it’s added by default.
yeah, it is.
hrmm
I am wondering if send private information is off by default.
looks like it is on by default.
James Connor
@megakid
Jan 18 2016 18:57
hm ok
Eric J. Smith
@ejsmith
Jan 18 2016 18:57
can you give me a link to one of your events?
James Connor
@megakid
Jan 18 2016 18:58
I am self hosted, trialing at the moment.
Eric J. Smith
@ejsmith
Jan 18 2016 18:58
ahh
ok
it should be there.
strange.
Blake Niemyjski
@niemyjski
Jan 18 2016 19:00
think that get needs to apply project data exclusions??
James Connor
@megakid
Jan 18 2016 19:00
ok i will investigate more.
Eric J. Smith
@ejsmith
Jan 18 2016 19:00
no don’t think so, they are explicitly adding things.
but I guess server side things can change.
Blake Niemyjski
@niemyjski
Jan 18 2016 19:00
ok, we are working towards releasing a new client hopefully very very soon which has fixes for when we send the environment username
yeah
so should I check them?
Eric J. Smith
@ejsmith
Jan 18 2016 19:01
what fixes does it have for username?
yeah, I guess that makes sense since there can be old code sending things that they decided to remove on the server side.
as far as the authtoken thing in the url… the controller action won’t use it, but our auth handler will look for certain patterns in the url and grab the token out and try it.
what that does is make it so that people can send a very simple get request to a url and be done.
which will be perfect for what I want to do with reporting benchmark results.
Blake Niemyjski
@niemyjski
Jan 18 2016 19:03
we changed it up so we don’t set the default user from environment user name, we also have fixes in where wer were overriding the PrivateInformation Setting
Eric J. Smith
@ejsmith
Jan 18 2016 19:03
will be incredibly simple for users to send random things.
Blake Niemyjski
@niemyjski
Jan 18 2016 19:03
and other changes
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 19:04
ok, so maybe @megakid needs to pull from our current dev branch and try it.
Blake Niemyjski
@niemyjski
Jan 18 2016 19:04
yeah, that would be best I need to review these changes and get them out the door (I hope today)
@ejsmith our auth handler doesn’t look at routes nor should it
just specify a querystring key with your access key and your done
James Connor
@megakid
Jan 18 2016 19:10
Thanks guys. Sounds like it's not expected to be there in the latest release? I can try your Dev branch if you think it'll help you
Blake Niemyjski
@niemyjski
Jan 18 2016 19:21
it depends on the current release based on settings
we tried to set it but there was cases where the setting was being overridden in different integrations
we’ve also renamed the plugin that was setting it and dealing with it
be back in a bit
Sander Rijken
@srijken
Jan 18 2016 19:52
@niemyjski deduping.. looking as we speak
how sure are we about the innermost exception thing?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:01
well if the innermost exception doesn’t match nothing else will match
@ejsmith correct me if I’m wrong
for dedupling we could just do innermost message + type + stacktrace on the raw exception and if it matches we bail
Sander Rijken
@srijken
Jan 18 2016 20:02
makes sense I think
seems like Exception.Message is guaranteed non-null, and Exception.StackTrace can be null
Eric J. Smith
@ejsmith
Jan 18 2016 20:06
Client side dedupinng needs to be really simple. We don't want a bunch of logic there. Should just match exact same errors.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:06
yeah
but what I’m asking is can we get away with only checking the innermost exception
if it doesn’t match nothing else will ever match
right now we walk all of them
Sander Rijken
@srijken
Jan 18 2016 20:07
if it does match, we have to check the one on top of that
Blake Niemyjski
@niemyjski
Jan 18 2016 20:07
yeah
Sander Rijken
@srijken
Jan 18 2016 20:07
because 2 different calls can cause the same nullref for instance
Blake Niemyjski
@niemyjski
Jan 18 2016 20:07
good point
Sander Rijken
@srijken
Jan 18 2016 20:08
but walking outwards isn’t trivial
so it’s easier to just walk all
other gotcha is AggregateException
Blake Niemyjski
@niemyjski
Jan 18 2016 20:08
yeah
we really need to figure something out for that
well we could just walk all
Sander Rijken
@srijken
Jan 18 2016 20:09
.Flatten() it, and walk the list in case of aggregate
is aggregate guaranteed to be the outer? probably not.. :/
Blake Niemyjski
@niemyjski
Jan 18 2016 20:09
snag in your plan is pcl doesn’t have aggregate exceptions
so you can’t call flatten :\ unless you use reflection
I think one thing were going to have to do is break out aggregate exceptions into x exceptions
Sander Rijken
@srijken
Jan 18 2016 20:10
ugh
Blake Niemyjski
@niemyjski
Jan 18 2016 20:10
would make everything easier
Sander Rijken
@srijken
Jan 18 2016 20:10
so I can also not type-check for aggregate?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:10
nope
that type doesn’t exist in pcl
it does and it doesn’t they just didn’t include it
Sander Rijken
@srijken
Jan 18 2016 20:11
we could call .ToString()
that has a special overload for AggregateException
but I don’t like it
Blake Niemyjski
@niemyjski
Jan 18 2016 20:11
me either
Sander Rijken
@srijken
Jan 18 2016 20:12
don’t want reflection either
Blake Niemyjski
@niemyjski
Jan 18 2016 20:12
nope
I say we just do inner exception if it’s an aggregate it won’t work right but it will at the same time
Sander Rijken
@srijken
Jan 18 2016 20:12
it’ll take the first inner and walk that right
Blake Niemyjski
@niemyjski
Jan 18 2016 20:12
well kinda curious which one it would pick to match on and if there is two different exceptions one might get discarded or not
yeah
Frank Ebersoll
@frankebersoll
Jan 18 2016 20:15
can’t we walk outwards by doing recursion?
Sander Rijken
@srijken
Jan 18 2016 20:15
sure
except for the other exceptional one..
oh nevermind
Frank Ebersoll
@frankebersoll
Jan 18 2016 20:16
why not?
Sander Rijken
@srijken
Jan 18 2016 20:16
I was thinking stackoverflow, but that’s nonsense
AggregateException remains the only real problem then
Frank Ebersoll
@frankebersoll
Jan 18 2016 20:17
what about „GetBaseException“?
in .net, this should return the innermost exception
Sander Rijken
@srijken
Jan 18 2016 20:17
no
it returns the first one that has innerexceptions
“Returns the first inner AggregateException that contains more or less than one inner exception"
Frank Ebersoll
@frankebersoll
Jan 18 2016 20:18
uh-oh
Blake Niemyjski
@niemyjski
Jan 18 2016 20:18
yeah
we have a helper method in the .net client to try and get the innermost exception
Sander Rijken
@srijken
Jan 18 2016 20:19
yeah but that one doesn’t help when you want to move outwards again
Blake Niemyjski
@niemyjski
Jan 18 2016 20:19
maybe just update what we do right now
but use the hash of those three properties
Sander Rijken
@srijken
Jan 18 2016 20:19
MSDN shows it AE as supported btw?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:19
last time I tried it doesn’t exist in our pcl profile
Sander Rijken
@srijken
Jan 18 2016 20:20
hm
how can I be sure of that?
if I try to use it, it compiles fine
I think the compiler checks it?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:21
yeah
the compiler will check it
Sander Rijken
@srijken
Jan 18 2016 20:22
think there’s no problem then
Eric J. Smith
@ejsmith
Jan 18 2016 20:24
we should be doing this on the error model after the exception is already shredded into the model. Then we don’t need reflection and we can use it no matter how the error model is populated.
Sander Rijken
@srijken
Jan 18 2016 20:24
true, but there the stack is parsed into objects per stack frame
so it’ll be more expensive
Eric J. Smith
@ejsmith
Jan 18 2016 20:26
it will be less expensive than using reflection. :-)
we are already using reflection to pull off all of the unknown properties on an exception.
Sander Rijken
@srijken
Jan 18 2016 20:27
true
Eric J. Smith
@ejsmith
Jan 18 2016 20:27
we need to think about this and what we are trying to accomplish. We don’t want the client having logic that gets outdated and then we need everyone to update their clients.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:27
@ejsmith if we use the raw stacktrace string gethashcode is going to be insanely fast
Eric J. Smith
@ejsmith
Jan 18 2016 20:28
we really just want to limit the number of times that we are sending the same error over and over… the problem lies in the definition of “the same"
yeah, we should be using gethashcode on the strings for sure.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:28
but I’m saying we could just do that for the dup checker and be done
no shredding
could even put it before the error model shredder and that would save a bunch of work
Eric J. Smith
@ejsmith
Jan 18 2016 20:29
what do you guys think about keeping track of the number of occurences that we are discarding and then setting the value property on the event that we let through?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:29
would be nice but lots of extra work
Eric J. Smith
@ejsmith
Jan 18 2016 20:29
then you could know how many of this error are really happening and how many are being thrown away.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:29
well I guess not too much work
Eric J. Smith
@ejsmith
Jan 18 2016 20:30
we are already storing the fact that we have sent this error some place.
and a time since the last time we sent it.
so we know which ones to discard
Blake Niemyjski
@niemyjski
Jan 18 2016 20:30
but all of our stats come from actual occurrences so how would that be aggregated
Eric J. Smith
@ejsmith
Jan 18 2016 20:30
so it’s just a matter of changing that to a counter that we increment when we throw them away and then resetting it when we let one through.
Sander Rijken
@srijken
Jan 18 2016 20:31
most of the work is in .UI, if you want to show it
Eric J. Smith
@ejsmith
Jan 18 2016 20:31
well, we are already talking about adding an option to do a chart for value… so we could use that for this.
but we would have to default the events without a value to 1.
would be super nice to know this information though.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:31
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 20:31
without the event spam.
I really like that idea.
Sander Rijken
@srijken
Jan 18 2016 20:34
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 20:34
these are things that are helping people get more value out of the system with less spam and less cost.
for both us and for the user.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:34
+1
Eric J. Smith
@ejsmith
Jan 18 2016 20:35
so there is really only 1 question… what is the definition of duplicate that is dumb enough that we won’t have to keep changing it?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:37
eh that could be hard because tags / users could change
etc
Eric J. Smith
@ejsmith
Jan 18 2016 20:37
yeah, that is a good point.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:38
same user and same stacktrace + message + type
Eric J. Smith
@ejsmith
Jan 18 2016 20:38
we really need a way of cheaply calculating a hashcode for the entire event.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:38
guess tags don’t matter
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 20:38
hrmm
Blake Niemyjski
@niemyjski
Jan 18 2016 20:38
minus dates
Eric J. Smith
@ejsmith
Jan 18 2016 20:38
this could be really useful in general… like feature usage stuff too…
Blake Niemyjski
@niemyjski
Jan 18 2016 20:38
would be cool if you could bail calculating the hashcode on the first mismatch
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 20:39
say you are sending a crap load of feature usages… we could dedupe those and just set value on 1 of them.
Sander Rijken
@srijken
Jan 18 2016 20:39
sounds great
Blake Niemyjski
@niemyjski
Jan 18 2016 20:39
doesn’t really work for metrics
Eric J. Smith
@ejsmith
Jan 18 2016 20:39
why not?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:39
because for some we are setting the value to the time it took or some time
Eric J. Smith
@ejsmith
Jan 18 2016 20:39
works awesome for counters
Blake Niemyjski
@niemyjski
Jan 18 2016 20:39
can’t really aggrated them
yeah but not for page load time
Eric J. Smith
@ejsmith
Jan 18 2016 20:39
yes, which would be a diff event.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:39
lol
what about debugging log messages
Sander Rijken
@srijken
Jan 18 2016 20:40
same thing?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:40
you might never want them deduplicated
Eric J. Smith
@ejsmith
Jan 18 2016 20:40
again, it would be a different event.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:40
I might have a condition and I want to look to see if it’s hit x times
Sander Rijken
@srijken
Jan 18 2016 20:40
if it’s the exact same message, and it says, ‘this happened 50 times'
Blake Niemyjski
@niemyjski
Jan 18 2016 20:40
yeah
guess so
Sander Rijken
@srijken
Jan 18 2016 20:40
kinda like the debug console in chrome does
Blake Niemyjski
@niemyjski
Jan 18 2016 20:41
but for chronological order
it’s not 1 to 1
Eric J. Smith
@ejsmith
Jan 18 2016 20:41
well, we can’t change their message… but we can roll up things with the exact same message inside of a small time window.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:41
most cases it would be awesome but there are a few use cases where your compairing our logs to your code to see execution flow
Eric J. Smith
@ejsmith
Jan 18 2016 20:41
you have to remember that this is a small window of time.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:41
guess your debug messages better not be the same
Sander Rijken
@srijken
Jan 18 2016 20:41
remember that currently this only happens for a 2 second timeframe
Eric J. Smith
@ejsmith
Jan 18 2016 20:41
exactly
this is just to prevent massive spam
Blake Niemyjski
@niemyjski
Jan 18 2016 20:41
yeah
Sander Rijken
@srijken
Jan 18 2016 20:43
I kinda have this thing worked out on Exception, but looks like that might not be the ultimate way to go now ;-)
Eric J. Smith
@ejsmith
Jan 18 2016 20:44
so if we just implement gethashcode on these objects then we can do this generically.
because we need to go through the list of data items and check them too.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:44
@srijken has a lot of experience with this and he says it’s slow as heck
Eric J. Smith
@ejsmith
Jan 18 2016 20:44
what is slow?
Sander Rijken
@srijken
Jan 18 2016 20:44
huh?
Eric J. Smith
@ejsmith
Jan 18 2016 20:44
gethashcode?
Sander Rijken
@srijken
Jan 18 2016 20:44
what’s slow?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:44
calling gethashcode a ton
I think the less properties we check the best
Eric J. Smith
@ejsmith
Jan 18 2016 20:45
I think that depends on what you are doing inside of gethashcode.
Sander Rijken
@srijken
Jan 18 2016 20:45
there’s a difference between a ton, and millions of times
Blake Niemyjski
@niemyjski
Jan 18 2016 20:45
there is somethings we dont’ care about like target site etc..
yeah
Sander Rijken
@srijken
Jan 18 2016 20:45
@ejsmith it was string.GetHashCode() that showed up in the profiler :)
if you calculate 5 million of them, it still takes a little while.. I wouldn’t call it slow as heck though
Eric J. Smith
@ejsmith
Jan 18 2016 20:45
right, but if you are overriding gethashcode then you pick the things that matter to include.
Sander Rijken
@srijken
Jan 18 2016 20:45
I agree
Eric J. Smith
@ejsmith
Jan 18 2016 20:46
yeah, it’s a microbenchmark thing… probb
probably expensive doing gethashcode on a really large string.
Sander Rijken
@srijken
Jan 18 2016 20:46
the only thing that was holding me back a little bit is that from that point on, they truly are the same.. not similar, but the same
Blake Niemyjski
@niemyjski
Jan 18 2016 20:46
yeah
Eric J. Smith
@ejsmith
Jan 18 2016 20:46
yeah, which I think is fine.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:46
what would be badass
Eric J. Smith
@ejsmith
Jan 18 2016 20:46
I don’t think we are using equality on them anywhere.
Blake Niemyjski
@niemyjski
Jan 18 2016 20:46
if we could do it by level
first class properties check the hashcode and bail if they don’t match then go deeper
Sander Rijken
@srijken
Jan 18 2016 20:47
dictionary/hashset or anything like that?
Blake Niemyjski
@niemyjski
Jan 18 2016 20:47
is that even possible in .net?
isn’t it all or nothing
Sander Rijken
@srijken
Jan 18 2016 20:47
you calculate the hashcode, period
Blake Niemyjski
@niemyjski
Jan 18 2016 20:47
yeah
Sander Rijken
@srijken
Jan 18 2016 20:47
and the result of that isn’t supposed to change
Eric J. Smith
@ejsmith
Jan 18 2016 20:47
yeah
Blake Niemyjski
@niemyjski
Jan 18 2016 20:47
if you could calculate parts of it that would be sweet.. but guess you can't
Eric J. Smith
@ejsmith
Jan 18 2016 20:47
I think it would be good.
we would pick the pieces that we think are involved in equality.
and use a hashcode combiner to combine all the hashcodes together.
Sander Rijken
@srijken
Jan 18 2016 20:49
sounds good to me
Eric J. Smith
@ejsmith
Jan 18 2016 20:50
so the we are expanding the scope for this to apply to all types of events and keeping track of number of discarded dupes and setting the value property to include the number of discarded events.
the cool thing is if we had the ICacheClient thing going then this could be something that is deduped across your entire web farm and other processes.
Sander Rijken
@srijken
Jan 18 2016 20:52
ugh how do you generate the equals stuff again?
Eric J. Smith
@ejsmith
Jan 18 2016 20:53
resharper, edit, generate code...
Sander Rijken
@srijken
Jan 18 2016 20:53
thx
Eric J. Smith
@ejsmith
Jan 18 2016 20:53
I just have to say that this is really awesome. :-) Having an open source project and getting more people involved. Fun!
Sander Rijken
@srijken
Jan 18 2016 20:55
this question.. when is something equal, will come up a few more times I guess
Eric J. Smith
@ejsmith
Jan 18 2016 20:56
yeah
it will
hopefully we can get it right.
Sander Rijken
@srijken
Jan 18 2016 20:57
so we want to have this on the Event level right
Eric J. Smith
@ejsmith
Jan 18 2016 20:57
yeah
Sander Rijken
@srijken
Jan 18 2016 20:58
getting hard right away… you kinda have to choose what properties in the data dictionary you do and don’t compare
Eric J. Smith
@ejsmith
Jan 18 2016 20:58
well, I would think that we would just use gethashcode on each of them and combine all of those hashcodes.
I think there may be known data names that we may exclude, but everything else would be included.
Sander Rijken
@srijken
Jan 18 2016 21:00
right that might work
Eric J. Smith
@ejsmith
Jan 18 2016 21:00
don’t even know of any known ones that we would exclude off the top of my head.
Sander Rijken
@srijken
Jan 18 2016 21:01
I think @trace and @environment, looking at things
Eric J. Smith
@ejsmith
Jan 18 2016 21:01
yeah, @environment makes sense.
well
Sander Rijken
@srijken
Jan 18 2016 21:01
don’t want to mark is a different because mem-free changed or something
Blake Niemyjski
@niemyjski
Jan 18 2016 21:01
we want environment
Eric J. Smith
@ejsmith
Jan 18 2016 21:01
maybe not… if the environment info model didn’t include some of it’s properties.
Blake Niemyjski
@niemyjski
Jan 18 2016 21:01
we use it to see what machines are affected
yeah
like we don’t care about the memory
Sander Rijken
@srijken
Jan 18 2016 21:02
indeed
but then GetHashCode is getting hairy again..
Eric J. Smith
@ejsmith
Jan 18 2016 21:02
well, I think personally… I’d rather have my events deduped and allow more events to be processed than I would care about knowing which machine it happened on.
Blake Niemyjski
@niemyjski
Jan 18 2016 21:03
yeah
but at the same time it shouldn’t come through just because the amount of available memory changed
Eric J. Smith
@ejsmith
Jan 18 2016 21:03
right
so each one of these models is going to have to make decisions about what is important and what isn’t,
Sander Rijken
@srijken
Jan 18 2016 21:03
yeah
Blake Niemyjski
@niemyjski
Jan 18 2016 21:04
I’d be down for not checking any extended data
on any of the models
Eric J. Smith
@ejsmith
Jan 18 2016 21:04
but they all implement gethashcode and the event just calls gethashcode on all of them and combines their hashcodes.
Blake Niemyjski
@niemyjski
Jan 18 2016 21:04
yeah
Sander Rijken
@srijken
Jan 18 2016 21:04
might be nasty
Eric J. Smith
@ejsmith
Jan 18 2016 21:04
what the heck? if we don’t include any extended data that would be throwing stuff away.
Sander Rijken
@srijken
Jan 18 2016 21:04
let me think to come up with why that might be a problem
Eric J. Smith
@ejsmith
Jan 18 2016 21:04
we want exact matches in the sense of everything that we care about.
Sander Rijken
@srijken
Jan 18 2016 21:05
maybe I should look at one of these things we’re trying to prevent, and diff a few responses?
Blake Niemyjski
@niemyjski
Jan 18 2016 21:05
I think we need to check things that make sense, anything in extended data on an exception or root of the event makes sense, other than that doesn't
Eric J. Smith
@ejsmith
Jan 18 2016 21:05
yeah
everything makes sense… it’s got to be dumb… if people are adding custom objects onto these events then we don’t want those getting thrown away.
unless they override gethashcode and decide what matters too.
and if they really don’t care, then they could return the same hashcode always.
but we want it to just work without them knowing or caring.
Sander Rijken
@srijken
Jan 18 2016 21:07
if something is added.. it should automatically “matter"
Eric J. Smith
@ejsmith
Jan 18 2016 21:07
and that includes making sure that all of the data that they are adding does not get thrown away.
exactly
Blake Niemyjski
@niemyjski
Jan 18 2016 21:07
yeah but my issue is if they don’t implement gethashcode
Eric J. Smith
@ejsmith
Jan 18 2016 21:08
that is fine
Blake Niemyjski
@niemyjski
Jan 18 2016 21:08
it will change on every instance even if it’s the same values
Eric J. Smith
@ejsmith
Jan 18 2016 21:08
it will never be a dupe
yes and that is fine
Sander Rijken
@srijken
Jan 18 2016 21:08
that’s kinda what happens now :)
Blake Niemyjski
@niemyjski
Jan 18 2016 21:09
yeah
I’ll let you guys be :D
Eric J. Smith
@ejsmith
Jan 18 2016 21:09
simple errors will be dupes
Blake Niemyjski
@niemyjski
Jan 18 2016 21:09
I need to get this stuff pushed
Eric J. Smith
@ejsmith
Jan 18 2016 21:09
simple events will be dupes
Sander Rijken
@srijken
Jan 18 2016 21:10
one thing that really doesn’t matter is the value that indicates how many dupes were left out, that shouldn’t be in the hashcode ever
Eric J. Smith
@ejsmith
Jan 18 2016 21:11
well if that was populated on the event, then we may not it to not be a dupe… normally that won’t be populated.
Sander Rijken
@srijken
Jan 18 2016 21:11
right
Eric J. Smith
@ejsmith
Jan 18 2016 21:11
but for things like page views, we are populating value with the amount of time it takes to load the page.
Sander Rijken
@srijken
Jan 18 2016 21:11
I checked something that should be dupes
only thing changed: id and date
Blake Niemyjski
@niemyjski
Jan 18 2016 21:11
we don’t want to ever check dates
Eric J. Smith
@ejsmith
Jan 18 2016 21:12
yeah, date has to be excluded
and id too
Blake Niemyjski
@niemyjski
Jan 18 2016 21:12
well
what id property
there shouldn’t be any id properties
Sander Rijken
@srijken
Jan 18 2016 21:12
sure.. but I was just checking what was different in the json :)
Eric J. Smith
@ejsmith
Jan 18 2016 21:12
ahh yeah
so it would work without those being in there?
Sander Rijken
@srijken
Jan 18 2016 21:13
if you throw those 2 out and do a diff, they’re identical
can’t throw that diff public, might contain slightly sensitive info
oh wait
referenceid (makes sense)
and trace info is different
Eric J. Smith
@ejsmith
Jan 18 2016 21:14
yeah, it does.
yeah, we should toss trace.
eventualy, we won’t need trace being on errors.
since you can send us your log messages as separate events. :-)
Sander Rijken
@srijken
Jan 18 2016 21:15
yeah
I like how low friction that thing is though
and it also works for 3rd party traces
different story :)
Eric J. Smith
@ejsmith
Jan 18 2016 21:16
yeah
:-)
Sander Rijken
@srijken
Jan 18 2016 21:19
looks like if we get rid of trace in the data dictionary, that’s enough
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:22
i don’t know if you already have come up with a solution, but what I didn’t understand yet is how you want to assign the counter to an existing stack
using the hashcode? then, the server would need the hashcode and find that event
Eric J. Smith
@ejsmith
Jan 18 2016 21:24
it’s not on a stack… we are deduping client side and then when we let one of the events through, we just set the value property on the event to the number of times it happened.
server side will still stack it based on the error info.
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:24
okay
so we can, ehm… we can do throttling!
Sander Rijken
@srijken
Jan 18 2016 21:25
uhm there’s another problem with that approach
we let the first through
then skip 1000
what if #1002 doesn’t come?
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:25
we can add a timer
Sander Rijken
@srijken
Jan 18 2016 21:26
how does that help?
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:26
we can just put the invariant „one specific event will only be sent once per minute"
so, send the first event immediately
Sander Rijken
@srijken
Jan 18 2016 21:26
that’s in place (except the time is 2 seconds)
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:26
send the second event after 1 minute, and count up if other events follow in between
Eric J. Smith
@ejsmith
Jan 18 2016 21:27
yeah, even if its not be perfect and we lose some counts. I am ok with that.
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:27
why will we lose counts?
Sander Rijken
@srijken
Jan 18 2016 21:27
it batches every 2 seconds now
but there’s a chance there’s no trigger for the last one
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:27
yes, but we can just add that.
Sander Rijken
@srijken
Jan 18 2016 21:28
sure
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:28
why don’t we send them anyways, just wait for 2 seconds?
it would need a timer, and that would send every „buffered“ event together with its counters
Sander Rijken
@srijken
Jan 18 2016 21:29
because.. we’re in the pipeline right? I can decide to let the event through or dump it
Eric J. Smith
@ejsmith
Jan 18 2016 21:29
yeah, we can do it in different ways.
Sander Rijken
@srijken
Jan 18 2016 21:29
I can’t re-insert it into the pipeline
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:30
but i can continue the pipeline, if we make it async
it isn’t though.
Sander Rijken
@srijken
Jan 18 2016 21:30
that sounds like a nice plan yeah
also like a breaking change though
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:31
hmm. it could be added as a feature
we could support both sync and async plugins
Eric J. Smith
@ejsmith
Jan 18 2016 21:31
we can get a reference to the client in the pipeline and send the event from a timer.
we are doing that in the heartbeat plugin.
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:32
in the dedup plugin, we could use a TaskCompletionSource and return the task and just complete it when the timer is due
Eric J. Smith
@ejsmith
Jan 18 2016 21:32
yeah, I wish the whole client was async. :-(
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:32
i don’t think this will have to be breaking
the code calling the plugins can just call the void (sync) or await the Task (async), depending on what interface the plugin implements
Eric J. Smith
@ejsmith
Jan 18 2016 21:37
I think we need to keep everything sync until we redo everything to be async.
Sander Rijken
@srijken
Jan 18 2016 21:37
so you have 2 plugin interfaces? the current one and a new one?
Eric J. Smith
@ejsmith
Jan 18 2016 21:37
which I would love to do soonish.
we can do the same thing in a timer.
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:37
IPlugin and IAsyncPlugin?
Sander Rijken
@srijken
Jan 18 2016 21:38
@ejsmith timer doesn’t work without async in this case..
Eric J. Smith
@ejsmith
Jan 18 2016 21:38
why?
Sander Rijken
@srijken
Jan 18 2016 21:39
because in a sync call, you can only decide to dump it or let it through
well you can’t do it in a way that nothing gets lost
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:39
you can re-enqueue it, though
Sander Rijken
@srijken
Jan 18 2016 21:39
you can?
Eric J. Smith
@ejsmith
Jan 18 2016 21:39
yeah, and that is fine… if we haven’t seen it before… let it though and increment a counter for the hashcode of the event...
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:39
well, you can make it a new event
Eric J. Smith
@ejsmith
Jan 18 2016 21:40
if we have seen it before, we increment the counter… then in a thread timer we send the discarded ones as a single event.
and we can do that on dispose too.
hrmm… other things might be shut down by then.
Sander Rijken
@srijken
Jan 18 2016 21:40
right
Eric J. Smith
@ejsmith
Jan 18 2016 21:41
but if we end up losing some dupe counts… I am really not that upset.
so 1st event comes through… we let it go through...
2nd one comes and we put that in a background queue… 3rd one comes, we grab that event off the background queue and increment the value...
Sander Rijken
@srijken
Jan 18 2016 21:42
we could also do the simple way first, and then make it perfect when moving towards async
Eric J. Smith
@ejsmith
Jan 18 2016 21:42
timer gets called and we send the batched event.
Sander Rijken
@srijken
Jan 18 2016 21:43
VS has stopped working..
Eric J. Smith
@ejsmith
Jan 18 2016 21:43
we really need a PCL concurrentdictionary
dude, VS has been crashing on me left and right.
Sander Rijken
@srijken
Jan 18 2016 21:44
I have love/hate relationship with resharper… I suspect it, and also for slowing down
Eric J. Smith
@ejsmith
Jan 18 2016 21:44
yeah, I feel the same about VS as a whole.
I want a lightweight editor.
Sander Rijken
@srijken
Jan 18 2016 21:45
yeah but VS code is too light :)
Eric J. Smith
@ejsmith
Jan 18 2016 21:46
right now it is a little too light… but I think it’s damn close.
I want a good text editor, intellisense, debugging, and extensions.
nothing more.
Frank Ebersoll
@frankebersoll
Jan 18 2016 21:47
refactoring?
might come from extensions.
Eric J. Smith
@ejsmith
Jan 18 2016 21:47
yeah, but even that should be in extensions
so I can pick and choose what I want.
VS Code is not far off.
Sander Rijken
@srijken
Jan 18 2016 21:53
time to call it a day, I’ll work some more on this tomorrow
Eric J. Smith
@ejsmith
Jan 18 2016 21:54
ok, later man. Have a good night.
Blake Niemyjski
@niemyjski
Jan 18 2016 23:06

public setEventReference(name: string, id: string): EventBuilder {
if (!name) {
throw new Error('Invalid name');
}

if (!id ||!this.isValidIdentifier(id)) {
  throw new Error(`Id ${this._validIdentifierErrorMessage}`);
}

this.setProperty('@ref:' + name, id);
return this;

}

kinda sucks that it’s going to throw an exception rather than bail
but I guess it should
Eric J. Smith
@ejsmith
Jan 18 2016 23:06
yeah, it really should.
it’s the end user dev doing it.
we don’t want it to just do nothing.
Blake Niemyjski
@niemyjski
Jan 18 2016 23:07
yeah
.net client is pushed
server is pushed
Eric J. Smith
@ejsmith
Jan 18 2016 23:08
cool
Blake Niemyjski
@niemyjski
Jan 18 2016 23:35
JS client is pushed
Eric J. Smith
@ejsmith
Jan 18 2016 23:39
nice
Blake Niemyjski
@niemyjski
Jan 18 2016 23:41
UI is pushing :D
tomorrow is bug fixing day
and monitoring hehe
Eric J. Smith
@ejsmith
Jan 18 2016 23:42
nice
Blake Niemyjski
@niemyjski
Jan 18 2016 23:42
pray we don’t have to
if nothing big comes up we’ll deploy new server release on wednesday
I’m more comfy with this release because we’ve already been running the server side for weeks so it’s mostly any ui bugs that popup
and those are isolated into one module and have been tested pretty dang good