These are chat archives for exceptionless/Discuss

11th
Jun 2015
Lukas Wöhrl
@woehrl01
Jun 11 2015 13:59
hi there, is just upgrade from Exceptionless 1.x to 2.0.441. I can recive new exceptions successfully, but all my old ones are gone, is there a way to import them?
Blake Niemyjski
@niemyjski
Jun 11 2015 14:12
@woehrl01 Hey!
how did you upgrade
@woehrl01 did you run the migration jobs?
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:14
I downloaded the zip files (api + ui) from github
do I have to start them manually?
if that's the case, how can I do that?
I used the .yml from the github project for elastiscsearch and just adjusted the URLs and connectionstrings
Blake Niemyjski
@niemyjski
Jun 11 2015 14:16
Yes, run the jobs from App_Data\JobRunner
you'd need to run the StackMigrationJob, QueueEventMigrations, EventMigration
You can shell a job like: -t "Exceptionless.Core.Jobs.DailySummaryJob, Exceptionless.Core" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core"
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:19
Do I have to compile them by myself? because I can't find jobs, named like that
ups, sorry, looked, that the wrong directory
I only have "Exceptionless.EventMigration.dll" is this the one?
Blake Niemyjski
@niemyjski
Jun 11 2015 14:24
nope, the assemblies are all in there
yep
that contains all of the jobs required
It's a little bit of work..
to figure out the type names.
we are trying to make it a painless process in the future
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:25
C:\exceptionless.api\App_Data\JobRunner>Job.bat -t "Exceptionless.Core.Jobs.StackMigrationJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core"
this doesn't work
could you please tell me, whats possible wrong :)
Blake Niemyjski
@niemyjski
Jun 11 2015 14:26
Do Job.exe instead of bat
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:26
2015-06-11 16:25:15.6358|ERROR|TypeHelper|Unable to resolve type:
is the error
Blake Niemyjski
@niemyjski
Jun 11 2015 14:27
Exceptionless.EventMigration.StackMigrationJob
your type is wrong
any chance you could paste all the commands you used when you are done and I'll put it in a gist
Job.exe -t "Exceptionless.EventMigration.StackMigrationJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core"
Job.exe -t "Exceptionless.EventMigration.QueueEventMigrationsJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core"
Job.exe -t "Exceptionless.EventMigration.EventMigrationJob, Exceptionless.EventMigration" -s "Exceptionless.Core.Jobs.JobBootstrapper, Exceptionless.Core"
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:30

System.AggregateException: Mindestens ein Fehler ist aufgetreten. ---> System.Nu
llReferenceException: Der Objektverweis wurde nicht auf eine Objektinstanz festg
elegt.
bei Foundatio.Jobs.JobRunner.<RunAsync>d5.MoveNext()
--- Ende der internen Ausnahmestapelüberwachung ---
bei System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCancele
dExceptions)
bei System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotific ation) bei System.Threading.Tasks.Task1.get_Result()
bei Foundatio.JobRunner.Program.Main(String[] args) in c:\projects\foundatio\
src\JobRunner\Program.cs:Zeile 31.
---> (Interne Ausnahme #0) System.NullReferenceException: Der Objektverweis wurd
e nicht auf eine Objektinstanz festgelegt.
bei Foundatio.Jobs.JobRunner.<RunAsync>d
5.MoveNext()<---

2015-06-11 16:29:51.2693|ERROR|JobRunner|Unable to create job instance.
2015-06-11 16:29:51.3270|ERROR|Program|Job "EventMigrationJob" error: Der Objekt
verweis wurde nicht auf eine Objektinstanz festgelegt.

this happens on QueueEventMigrationsJob and EventMigrationJob, the first one is working fine
Blake Niemyjski
@niemyjski
Jun 11 2015 14:31
ah
not 100% sure
let me look
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:36
ok
Blake Niemyjski
@niemyjski
Jun 11 2015 14:42
ah
you don't need the -s on the end
did that help?
blob
I just ran them and they worked
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:44
yep, seems without -s its working, I'l report when they are finished
Blake Niemyjski
@niemyjski
Jun 11 2015 14:44
The thing is open the job.exe.config and set the redis connection string
I just thought about that
because you need to queue the migrations to someplace (default is in memory)
and that doesn't really work when you have multiple processes
@woehrl01 we just added the -s flag in the latest nightly hehe
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:46
I just executed those, but still no additional exceptions :-/
Blake Niemyjski
@niemyjski
Jun 11 2015 14:51
you'll need to configure redis
otherwise the queue event migrations job is just going to queue things to nothing
(in memory queue)
Did the stacks show up in your elastic search instance
Lukas Wöhrl
@woehrl01
Jun 11 2015 14:54
no, the stacks are not showing up
I just renabled redis but still no exceptionions
so. I changed job.exe.config to the same values aus web.config (mongodb/redis/elastic) and I reexecuted the statements above for StackMigrationJob, QueueEventMigrationsJob and EventMigrationJob
but, the old exceptions are still not showing up
any further suggestions?
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:15
I just enabled, the INFO logging level. and it prints the following: The daterange looks a little bit off?
2015-06-11 17:14:30.9660|INFO|JobRunner|Starting job type "EventMigrationJob" on machine "VM9-DEV"...
2015-06-11 17:14:31.1126|WARN|Bootstrapper|StatsD Metrics is NOT enabled.
2015-06-11 17:14:31.2572|WARN|Bootstrapper|Azure Storage is NOT enabled.
2015-06-11 17:14:32.8286|INFO|MigrationJobBase|Public IP: 79.255.159.7
2015-06-11 17:14:32.8579|INFO|EventMigrationJob|Processing event migration jobs
for date range: 2014-07-31T00:00:00.0000000+00:00-2014-07-31T12:00:00.0000000+00:00
2015-06-11 17:14:33.4149|INFO|EventMigrationJob|Finished processing event migration jobs for date range: 2014-07-31T00:00:00.0000000+00:00-2014-07-31T12:00:00.0000000+00:00
Blake Niemyjski
@niemyjski
Jun 11 2015 15:24
Yeah that is good it's doing 12 hour day increments
that looks good
anything in the log for migrating stacks
If you want I could jump on a remote meeting for a few minutes and help take a look
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:25
yeah, I cleared redis and did all of this again, and with info enabled I get:
2015-06-11 17:21:57.3987|INFO|StackMigrationJob|Migrating stacks 8.000 total 2.711/s...
2015-06-11 17:21:57.4192|INFO|StackMigrationJob|Migrating stacks 8.050 total 2.711/s... etc
looks good so far, except that it's not showing :D
Blake Niemyjski
@niemyjski
Jun 11 2015 15:26
we'll what are you using to look at your elastic search content
elastic hq plugin?
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:27
I have currently nothing installed to look at the content
just installed elasticsearch for exceptionless
go there and then enter in your connection string for elastic
as shown and then connect
it will show you all the indexs / records
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:35
looks, good, I have >8000 stacks-v1 about the same count like in mongodb
Blake Niemyjski
@niemyjski
Jun 11 2015 15:36
ok
so looks like those were migrated
now just need to migrate events
how many errors do you have?
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:36
none, inside of elastic
Blake Niemyjski
@niemyjski
Jun 11 2015 15:37
yes but in mongo
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:37
about 46k
Blake Niemyjski
@niemyjski
Jun 11 2015 15:37
ok
that should be a pretty quick migration
you should just have to queue them
and them run the migration job
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:38
it is just queueing from today
Blake Niemyjski
@niemyjski
Jun 11 2015 15:39
2015-06-11 17:14:32.8579|INFO|EventMigrationJob|Processing event migration jobs
for date range: 2014-07-31T00:00:00.0000000+00:00-2014-07-31T12:00:00.0000000+00:00
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:39
aaaaah, i see , i have to run the last one with -c
Blake Niemyjski
@niemyjski
Jun 11 2015 15:39
it looks like it started queueing from 11m ago
ahh yeah
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:40
:D
Blake Niemyjski
@niemyjski
Jun 11 2015 15:40
sorry :)
my bad
lol
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:40
thank you very much!
Blake Niemyjski
@niemyjski
Jun 11 2015 15:40
np
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:40
now they are flooding in
thanks a lot for your help! and a great piece of software!
Blake Niemyjski
@niemyjski
Jun 11 2015 15:41
the ui won't refresh for these events
coming in
due to there would be thousands / millions of notifications for it
but once it's done any events submitted via teh normal api will refesh the ui
was it pretty easy to setup the 2.0 project?
thanks :D
tell your friends about us :D
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:42
It took some time to figure out that I need to elasticsearch.yml from the repo
but the hardest part is the migration ;)
it would be nice, if there is a .bat to start this part
Blake Niemyjski
@niemyjski
Jun 11 2015 15:43
yeah
if you want to send me all the commands you used
I'll create a gist for it
we are going to move towards a wiki on github and we can add that there
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:43
and of course that you need redis
Blake Niemyjski
@niemyjski
Jun 11 2015 15:43
yeah didn't think of that until you ran into that
normally if you have a single node you don't need it
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:44
that's the reason, I disabled it first ;)
Blake Niemyjski
@niemyjski
Jun 11 2015 15:44
but you do when you queue events for migration (due to multiprocess)
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:44
for the gist, it's just the three statements you provided + "-c" on the last one
Blake Niemyjski
@niemyjski
Jun 11 2015 15:45
if you have it setup, I'd probably still use it due to it being more reliable to process restarts
you don't lose any queued data on reboot
yeah
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:45
yep, I think I will keep it
and the importent thing is to use job.exe instead of .bat, because the last one seams to replicate itself into it's subdirectory
Blake Niemyjski
@niemyjski
Jun 11 2015 15:47
yeah it does that for azure webjoobs
I'll create a gist
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:51
perfect
Fixed a few things hehe
just let us know if you have a ny questions or feedback
We'll have an upgrade process to 3.0 as well
and then it should be smooth sailing
we are working on getting rid of mongo right now
one less dependency
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:55
the gist looks good!
without mongodb would be great, but not that bad currently ;)
Blake Niemyjski
@niemyjski
Jun 11 2015 15:56
yeah that's what we are working towards
less dependencies and easier to self host
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:56
is there something in the new UI where I can search for all events exception the exceptions?
Blake Niemyjski
@niemyjski
Jun 11 2015 15:56
we hope it makes this project even more popular and gets people talking about it more :)
for all events exeception the exceptions?
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:57
ah, I mean all events which are not exceptions
I suggest exceptionless a lot
Blake Niemyjski
@niemyjski
Jun 11 2015 15:58
yeah
-type:error
should do it
Lukas Wöhrl
@woehrl01
Jun 11 2015 15:58
my boss and collegues are loving it, too ;)
we were able to reduce the error count by 80% which is pretty awesome I think :)
:thumbsup: -type:error
the migration finished, everything working fine. thank you!
I can stop the jobs now, can't I?
Blake Niemyjski
@niemyjski
Jun 11 2015 16:17
yep