These are chat archives for exceptionless/Discuss

5th
Oct 2015
Blake Niemyjski
@niemyjski
Oct 05 2015 15:25
hey
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:13
if we submit one and the same event twice, shouldn't the server ignore the second instance?
i'm thinking about implementing a persistence for events in javascript (node / browser), and there are situations where you don't know if an event has been successfully submitted.
i would rather submit the event a second time and have the server ignore the second call
Blake Niemyjski
@niemyjski
Oct 05 2015 18:18
nopr
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:18
nopr?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:19
nope, server side doesn’t do any de duplicating the client does and the javascript one doesn’t have anything right now
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:19
how is this done in other clients?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:19
it should be a plugin
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:19
ah! okay.
Blake Niemyjski
@niemyjski
Oct 05 2015 18:19
it only affects error events
brb deploying something :)
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:19
k
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:25
okay, but all plugins run before the event is enqueued. let's assume we have one, single event in the queue
the queue is processed and submission fails somehow.
should we retry the event? how can we make sure that the event is submitted exactly once?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:27
yes
that will happen automatically
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:27
how?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:27
brb in 1
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:29
well, that's the point - this only works if you get a valid response.
this is also true for .NET - if there is no valid response, you don't know if it worked or if it didn't work.
the event could have been submitted or it could have been lost. you don't always know for sure.
Blake Niemyjski
@niemyjski
Oct 05 2015 18:30
if it’s not a successful status code then it wasn’t submitted
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:30
do you know for sure?
what if the network disconnects right after the server received the event?
i ask for a reason: there is this new sendBeacon API in JavaScript
Blake Niemyjski
@niemyjski
Oct 05 2015 18:31
then it will be retried if the client never got the status code.
sendBeacon will be the API to send diagnostics and tracking stuff from the browser to the server.
it works in the window.unload handler, too. and there is no means of getting any response. it's fire-and-forget
Blake Niemyjski
@niemyjski
Oct 05 2015 18:34
yeah that could work but it’s experimental
in that case we would just send it and remove the data
and hope it got sent
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:34
yes, of course...
but if not, on app restart, could we figure out if it was sent?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:35
from the client no
we could use reference ids
but the client doesn’t have access to getting an event by reference id
it can only get config and post an event
clients should be stupid
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:36
that's why I thought that some kind of idempotence on server side would be really cool
Blake Niemyjski
@niemyjski
Oct 05 2015 18:36
if a client drops an event it’s not the end of the world
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:36
submit(A) submit(A) should be the same as submit(A)
Blake Niemyjski
@niemyjski
Oct 05 2015 18:36
some data is better than nothing but it should be the goal to get all data
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:37
yes, it should be the goal - but duplicate data is not so smart.
Blake Niemyjski
@niemyjski
Oct 05 2015 18:37
if it’s a error object the you should only get 1 after we implement deduplicating.. but it’s only a duplicate within a small window
for the js/node client it was get it out there and get users using it
didn’t have time to do deduplicating right off the bat. but we do need it
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:38
yes, i understand. but I think it's a server thing :-)
de-duplicating on the client to prevent semantic duplicates
and de-duplicating on the server, to implement idempotent submission
Blake Niemyjski
@niemyjski
Oct 05 2015 18:39
when you do it server side you start having lots of issues
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:39
performance?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:39
app versions, performance, how do you know it came from one machine, clocks can be different etc..
list goes on
deduplicating an error should be the goal of the client instance..
then you have to implement logic for every version of every client
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:40
okay. hm.
Blake Niemyjski
@niemyjski
Oct 05 2015 18:40
like java / c# / javascript
cause it can do it using a in memory cache and it will be de duplicated using the same logic of that version
guess we try to keep it simple
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:41
okay.
Blake Niemyjski
@niemyjski
Oct 05 2015 18:41
you could do it either side but we feel its easier to do it client side
argggggggg <— hates async
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:42
async-await?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:43
yeah
and azure
hate that too right now
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:43
hate is a strong verb ;-)
Blake Niemyjski
@niemyjski
Oct 05 2015 18:44
did a ton of profiling on 3.1 and sped up a ton of things, made the site snappier.. loads instantly locally and seems to run great.. but soon as we start deploying.. azure takes forever to bring it up and then random lag everywhere
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:45
no azure experience over here, sorry.
aside from two VMs with VS2015
Blake Niemyjski
@niemyjski
Oct 05 2015 18:45
well azure == slow
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:45
yes, i figured that already.
Blake Niemyjski
@niemyjski
Oct 05 2015 18:46
what do you use?
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:46
for development?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:46
for prod
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:47
consulting only :-) no hosting.
that's all the fun, minus the hosting.
Blake Niemyjski
@niemyjski
Oct 05 2015 18:48
nice
so if you want to take a stab at deduplicating it should be a straight port from the C# one
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:48
but i'd also look into azure if I needed to host a site such as yours
Blake Niemyjski
@niemyjski
Oct 05 2015 18:50
yeah
makes it nice to scale
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:50
my question was not so much about deduplication, but rather about "reliable submission"
Blake Niemyjski
@niemyjski
Oct 05 2015 18:50
but there are times it makes us double take on our decision
if you get a 202 accepted it was submitted
if it’s not, it can’t be 100%
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:51
what would be needed to make it 100%?
Blake Niemyjski
@niemyjski
Oct 05 2015 18:51
perfect up to date browser and a connection at the datacentet
we auto rely it up to 3 times
and we wait
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:52
most of the time, 202 would be sufficient
Blake Niemyjski
@niemyjski
Oct 05 2015 18:52
retry*
if you’re seeing a case where its not retried let me know
I can pair with you as well
I can meet up and talk through this with you
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:57
okay, I first put some effort in the node client (onUncaughtException)
Blake Niemyjski
@niemyjski
Oct 05 2015 18:57
ok
thanks :)
Frank Ebersoll
@frankebersoll
Oct 05 2015 18:59
then I put together something to visualize what I think could be a problem
then we talk about that :-)
Blake Niemyjski
@niemyjski
Oct 05 2015 19:00
sounds like a solid plan
Frank Ebersoll
@frankebersoll
Oct 05 2015 19:00
thanks for your time!
Blake Niemyjski
@niemyjski
Oct 05 2015 19:02
no problem