These are chat archives for exceptionless/Discuss

8th
Oct 2015
Blake Niemyjski
@niemyjski
Oct 08 2015 04:13
@frankebersoll :) awesome job man
really appreciate it
Frank Ebersoll
@frankebersoll
Oct 08 2015 06:10
Hey there! Thanks for the feedback. I'm on mobile, but I can still take tou through the changes from my mind.
It all starts in the uncaughtException-handler, where we enqueue and call exit(1), if needed. This will trigger the exit-handler, which leads to consistent behavior for whenever the app exits,
regardless the reason.
The exit-handler calls process
Frank Ebersoll
@frankebersoll
Oct 08 2015 06:51
on the queue after setting its state to isApplicationExiting. I don't really like it, but I like it better than setting this state in the config (as it isn't really a setting but rather the state of the application) and I didn't want to refactor all signatures of the queue and the submission clients. Still, it's global state and that's not really good design.
I could see different things happen in the ExitController, depending on what the user has configured: We could try to submit all events in a for-loop, if they outnumber the batch size. We could filter the events so there are only critical ones left in the queue, so the application exits faster.
Frank Ebersoll
@frankebersoll
Oct 08 2015 07:29
on the queue
When the NodeSubmissionClient finds out that isApplicationExiting, it spawns a child process. This is pretty fast, as we only load a minimum of resources, see
Frank Ebersoll
@frankebersoll
Oct 08 2015 07:40
submitSync.ts. This reuses the http client code from before, now refactored into NodeSubmissionRequest.ts.
Frank Ebersoll
@frankebersoll
Oct 08 2015 07:46
We use two pipes (stdin and stdout) to communicate with the child process. Our request information is serialized as json and set as input of the child process, making it available at stdin. In the child, we pipe stdin to a writable stream, where the json arrives in chunks as Buffer. We push those chunks to an array and wait for the stream to end (on finish). Then, we join the chunks and deserialize the request info.
Important: we can do asynchronous stuff in the child process! The streaming of the input data and the http submission itself are both async - but our parent process waits for the child to finish.
Frank Ebersoll
@frankebersoll
Oct 08 2015 07:52
Having received a response from the server, this gets equally serialized and written to stdout, making it available as buffer in the parent process.
That's about it.
Frank Ebersoll
@frankebersoll
Oct 08 2015 07:58
On my way I found two issues in the infrastructure: One in node.d.ts that I fixes temporarily in the exceptionless code base and permanently borisyankov/DefinitelyTyped#6185
and one in tsproject that I didn't get my head around yet, regarding reuse of typescript module names that already get referenced in the typings.
Blake Niemyjski
@niemyjski
Oct 08 2015 13:30
yeah
Thanks for the explination
if you want we can pair up on working the isApplicationExisting to where it feels natural like you said
it’s almost like the whole app could benefit from it as if the app is shutting down you may want to cancel timers and things like that
also, I just thought of something..
the on exit… I think process queue only does one batch of items and then waits for the next call.. I’d need to look into this...
but if that’s the case we might not be sending everything but that’s fine.
great job tho
:-)
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:34
Thanks!
Blake Niemyjski
@niemyjski
Oct 08 2015 13:34
did you see my comment about behaviors
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:34
you're right, we send only one batch
Blake Niemyjski
@niemyjski
Oct 08 2015 13:34
I’m not 100% sure we need them.
since we only have in memory storage we may want to figure that out but baby steps
:)
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:35
yes, i always go for "do it when uou need it" or YAGNI
Blake Niemyjski
@niemyjski
Oct 08 2015 13:35
your solution made me think of something else.
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:36
yes, baby steps is the way to go. For now, we "usually" get unhandled exceptions.
I dont think we need to cancel timers or clean up - after the method exits, the app will terminate anyway
Blake Niemyjski
@niemyjski
Oct 08 2015 13:37
so other error reporting clients fall back to things like sending payloads in images.. and things like that.. I like having one client that’s taliored to just sending but it does seem like it would be nice to just swap them out if we detect a submission method would never work. in this case switching out the default node for a sync one on shutdown via updating the config object but… prob have race conditions and what we have works
yeah but what if a timer is running… and calls submit async.. I guess our config object will havei t and it will spawn multiple processes to submit
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:38
Node is single threaded
Blake Niemyjski
@niemyjski
Oct 08 2015 13:38
true
been dealing with async multi threaded shit for the past month :\
lol
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:38
at no time ever there will run code in parallel in one process
Blake Niemyjski
@niemyjski
Oct 08 2015 13:38
yeah this will work out good
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:38
i had to get used to it, too
Another thing: it's possible that the app crashes while queue.isProcessing is true
Blake Niemyjski
@niemyjski
Oct 08 2015 13:40
yeah
but that’s fine currently
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:40
then, we just let the queue do what it already does: queue.process will fail fast
Blake Niemyjski
@niemyjski
Oct 08 2015 13:40
because queue using in memory storage currently and we only ever remove the queued items after successful submit
but the app dies
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:41
in that case, our exception wont be submitted because we wouldnt know how far the current submission has gone
Blake Niemyjski
@niemyjski
Oct 08 2015 13:41
hmm
couldn’t we check the buffer tho
I see what you are saying hmm
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:42
No. Same "duplicate" discussion as yesterday 😉
Blake Niemyjski
@niemyjski
Oct 08 2015 13:42
if the process is shutting down that async callback is prob already running
I think we should just close. as long as it sent anything to the server.. we don’t care about the response..
yeah
lol
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:42
The async callback will never be called, because were in onExit
Blake Niemyjski
@niemyjski
Oct 08 2015 13:43
yeah, but all that callback does is remove events from the storage (which is in memory)
so we should just assume they were sent if IsProcessing
we could just ignore it too but at the possiblity of getting duplicate events
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:43
Its tricky. I havent looked into the server code yet.
Blake Niemyjski
@niemyjski
Oct 08 2015 13:44
if you want we can setup a time and we can cover all of this
I’d like to pull in these changes sooner than later :) too
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:44
As I said, I will make some graphics to talk about regarding those race conditions
Blake Niemyjski
@niemyjski
Oct 08 2015 13:44
ok
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:45
what has to be done next from your perspective? I like the project
i will send a pull request for documentation later, there are some things to update in md
Blake Niemyjski
@niemyjski
Oct 08 2015 13:47
from node or just exceptionless
exceptionless/Exceptionless.JavaScript#30 (see last comment)
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:48
I have no server here at home.... But were plannin to setup one at work, so I can contribute there, too
Blake Niemyjski
@niemyjski
Oct 08 2015 13:48
I’m not sure if we need that or not, I guess we need to have a discussion on that.. I’d rather have that discussion now becaus anything we change in the client like this we have to live with going forward (not that is a bad thing)
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:48
so, I'd like to work on the javascript client
Blake Niemyjski
@niemyjski
Oct 08 2015 13:49
next things, I need to tackle some of the issues on github.. we need a persisted storage implementation
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:49
Okay, discussion on what?
Blake Niemyjski
@niemyjski
Oct 08 2015 13:49
with a fallback
my last comment on the issue I just linked too
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:50
Yes, right. Browser first or node first?
Blake Niemyjski
@niemyjski
Oct 08 2015 13:50
Either but I’d like both :)
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:50
Your last comment? Sorry
Blake Niemyjski
@niemyjski
Oct 08 2015 13:50
exceptionless/Exceptionless.JavaScript#30
I left a long comment about behaviors we might want to support
like onexit onfreeze
I can only think of the onexit because browser should handle onfreeze for us
but you can integrate javascript into windows apps and they have events like that
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:52
As I said, I would do this later on when we need it. The most important thing would be onExit in the browser
i think
Blake Niemyjski
@niemyjski
Oct 08 2015 13:53
yeah
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:53
because when you navigate away or clise the tab your events are gone
Blake Niemyjski
@niemyjski
Oct 08 2015 13:53
so I think I remember seeing config.exitcontroller be injected
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:53
you said something about different submission methods and images?
Blake Niemyjski
@niemyjski
Oct 08 2015 13:54
I wonder if we should take that off the interface that gets passed to the constructor.. so if we want to change it later we won’t be breaking people
yeah, was just a food for thought.. I think we are fine now...
I’d like your thoughts, do you see people wanting to inject this?
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:56
i don't know. i would leave anything open for extension.
to be fair, I don't like the current service locator pattern very much
Blake Niemyjski
@niemyjski
Oct 08 2015 13:57
yeah, I’m fine with that and I love extension, just harder to break things later in javascript without adding more code
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:57
but most people do it like this in javascript...
I really like dependency injection like Ninject
Blake Niemyjski
@niemyjski
Oct 08 2015 13:57
yeah, we use simple injector
if you have a better way of doing it in javascript I’d love to see it
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:58
this would help making the tests much easier. our current test suite runs against the server
Blake Niemyjski
@niemyjski
Oct 08 2015 13:58
I’ve written a bunch of javascript.. but I’m still a nub
Frank Ebersoll
@frankebersoll
Oct 08 2015 13:59
because of the missing ability to mock anything, that's why you put if (200) else everywhere in the tests
Blake Niemyjski
@niemyjski
Oct 08 2015 13:59
yeah
I tried to componetize everything so it could make mocking / testing easier but I’m sure things could be lots better
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:00
but that's another subject :-) maybe I can look into that while adding functionality
example: NodeSubmissionRequest
Blake Niemyjski
@niemyjski
Oct 08 2015 14:01
so going forward on the commit do you think there is anything we should change before accepting.. you said you weren’t happy with isExitingApplication, I agree a bit, but am fine with it… I wonder if we should clean up the callbacks and change NodeSubmissionCallback to be ISubmissionCallback (it will add size to the ts def but nothing to the generated javascript.
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:01
this contains a method that abstracts sending of a request to the node server.
Blake Niemyjski
@niemyjski
Oct 08 2015 14:01
yeah
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:01
this could be mocked in all tests, so we could write real assertions
the timers!
you always need to abstract away timers.
Blake Niemyjski
@niemyjski
Oct 08 2015 14:02
would be nice to have just one submission client with the code that makes the request returns a normalized response
then we could just inject it and save some size
I tried doing that
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:02
in .NET i always have an ITimer interface around, so i can write a TestTimer : ITimer that has an .Advance(TimeSpan) method
that way, you can easily write tests
Blake Niemyjski
@niemyjski
Oct 08 2015 14:03
<— doesn’t normally write tests
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:03
ah :-)
okay, I currently try to establish test first at work
Damiaan
@dampee
Oct 08 2015 14:04
How do you connect foundatio to librato?
Blake Niemyjski
@niemyjski
Oct 08 2015 14:04
I’ve never went off code coverage or anything typically the things that get tested are core peices or to ensure we can take money / core things work
but I try to make sure people can extend / make testing easier if we needed it
@dampee
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:04
I don't test to make things tested - I write tests, because it changes how I think about the design upfront.
Damiaan
@dampee
Oct 08 2015 14:04
uhm. sorry to interrupt. Hi everyone.
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:05
hey there
we use the statsd implementation witch is crazy fast (sends via udp)
Damiaan
@dampee
Oct 08 2015 14:06
But that statsd is running locally?
otherwise, I would be able to log to this IP too...
Blake Niemyjski
@niemyjski
Oct 08 2015 14:08
no
we do it on linux
Damiaan
@dampee
Oct 08 2015 14:09
? then what is your setup? You advised me too look into foundatio and said you are using librato to gather everything. But were are you sending the metrics to?
@frankebersoll ==> I write tests, because it changes how I think about the design upfront. +1!!
Blake Niemyjski
@niemyjski
Oct 08 2015 14:11
@dampee We’re family so here is our guide I used to setup ours.. If you find any personal info let me know.. Also, if you’d be willing to convert these steps to a nice doc we can put on our wiki that would be greatly appreciated
@frankebersoll I sometimes do that, but typically we discuss ho the design should be and don’t go full implementation.. typically create the interfaces first and talk about it.
Damiaan
@dampee
Oct 08 2015 14:11
give me a second
Blake Niemyjski
@niemyjski
Oct 08 2015 14:17
ok
let me know how that works out for you
so basically we send a udp packet to a local node statsd server
which is insanely fast
Damiaan
@dampee
Oct 08 2015 14:18
oh. Ok. So you are running a local statsD with a librato backend. That was a missing piece. We can't invest into running a linux currently.
Blake Niemyjski
@niemyjski
Oct 08 2015 14:18
and then that server bulk sends metrics to librato
run node on windows?
you could always write a custom metrics client that talks straight to libreto but I think this method is faster?
Damiaan
@dampee
Oct 08 2015 14:19
Our capacity (time, people & customers) is just too limited for now. That's something to dream off...
Blake Niemyjski
@niemyjski
Oct 08 2015 14:19
@dampee I can relate but it will take you an hour to setup and then it works for ever
Damiaan
@dampee
Oct 08 2015 14:20
we don't have local infrastructure
that means extra cloud server
extra cloud server means more money to pay every month
Blake Niemyjski
@niemyjski
Oct 08 2015 14:20
you could write your own metrics client for foundatio.. would be super simple, you’d just have to implement the send logic which you could do with the http client
Damiaan
@dampee
Oct 08 2015 14:20
there's a student here looking into it :-)
Blake Niemyjski
@niemyjski
Oct 08 2015 14:20
yeah
could put this on a super micro linux vm :)
ok :)
if they have any questions have them jump on here
Damiaan
@dampee
Oct 08 2015 14:21
thanks a lot
Blake Niemyjski
@niemyjski
Oct 08 2015 14:21
if I don’t respond I’m deep into async profiling and will get back to you guys
Damiaan
@dampee
Oct 08 2015 14:21
goed luck
*good
:)
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:23
hah! that is super cool. didn't know that
CSLA?
i thought i was the only one to use that
Blake Niemyjski
@niemyjski
Oct 08 2015 14:23
it’s a framework
haha
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:24
do your performance work. i don't want to interrupt you.
haha yeah
I wrote all of those
Frank Ebersoll
@frankebersoll
Oct 08 2015 14:32
doesn't ReSharper Find Usages work for you?
I put it on a shortcut for that.
Blake Niemyjski
@niemyjski
Oct 08 2015 15:05
it does but it doesn’t work across controllers
any one here on the live stream and want to see how our deploymeents work
Frank Ebersoll
@frankebersoll
Oct 08 2015 15:08
yeah
bring it on
Blake Niemyjski
@niemyjski
Oct 08 2015 15:24
ok
Sander Rijken
@srijken
Oct 08 2015 22:25
the live stream?
oh missed that