These are chat archives for exceptionless/Discuss

5th
Nov 2015
Modesto San Juan
@modesto
Nov 05 2015 08:12
exceptionless/Exceptionless#153 hi, is there anything I can do to diagnose the problem. I'm testing it in a selfhosting instance and I've full access to the Elasticsearch server.
Blake Niemyjski
@niemyjski
Nov 05 2015 13:20
@modesto I’m taking a look into that now as well.
Blake Niemyjski
@niemyjski
Nov 05 2015 13:36
I can reproduce
wonder what it’s hitting the server as
@modesto are you familiar with elastic
Modesto San Juan
@modesto
Nov 05 2015 14:35
@niemyjski yes, but I'm a bit rusty with query syntax because it's been several months since my last project with elastic.
Blake Niemyjski
@niemyjski
Nov 05 2015 14:36
I’m writing some emails and then I’ll take a closer look
what I’d do is take a look at what was inserted into the idx field. That is what is indexed.
I’d then look at fiddler at what query was being executed and then run it directly in sense.
That’s what I’m going to do unless you beat me to it
Modesto San Juan
@modesto
Nov 05 2015 19:26
ok, I'm doing it rigth now.
Modesto San Juan
@modesto
Nov 05 2015 19:41
ok, as far as I see, if I query a range like data.age(>30 AND >=40), the query is : "query": "fixed:false hidden:false project:563ba4dc8840731f4c9b3dfa data.age:(>30 AND <=40)"
but if I query data.age:34, the ES query is "query": "fixed:false hidden:false project:563ba4dc8840731f4c9b3dfa idx.age-n:34"
I've do some trace and the problem seems to be in QueryProcessor.Process
Blake Niemyjski
@niemyjski
Nov 05 2015 19:43
hmm
that could very well be
so does
idx.age-n:(>30 AND <=40) work?
Modesto San Juan
@modesto
Nov 05 2015 19:46
in Sense ? yes!
Blake Niemyjski
@niemyjski
Nov 05 2015 19:46
ok
We have a test very similuar to that
I added your case to it and it parsed correctly
tells me it’s not our lucene query parser.
Could still be our parser that sits on top of it
Modesto San Juan
@modesto
Nov 05 2015 19:51
with ag3:34, at this line string expandedQuery = validator.UsesDataFields ? GenerateQueryVisitor.Run(result) : query;
Blake Niemyjski
@niemyjski
Nov 05 2015 19:51
I just finished up something and I’m going to run to lunch. I’ll take a look under the debugger
Modesto San Juan
@modesto
Nov 05 2015 19:51
validator.UsesDataFields is true
Blake Niemyjski
@niemyjski
Nov 05 2015 19:51
yeah
let me look
Modesto San Juan
@modesto
Nov 05 2015 19:51
but with age:(>30 AND <=40), is false
I'm going to dine, I'll also give it a try later
Blake Niemyjski
@niemyjski
Nov 05 2015 19:52
what line is that on
Modesto San Juan
@modesto
Nov 05 2015 19:52
Exceptionless.Core.Filter.QueryProcessor.Process
Blake Niemyjski
@niemyjski
Nov 05 2015 19:52
ok
one second I see that
that will be set to false
try data.age:(>30 AND <=40)
that we will only translate the query if it’s data.age
I’ll step through it
we should have someunit tests for our QueryProcessor
Modesto San Juan
@modesto
Nov 05 2015 20:06
If I debug this line, after stop in term "project", it stops in term with null Field and "Term" >30
and the same for the term <=40 , the Field is null
Definitely I have to go to dinner!
Blake Niemyjski
@niemyjski
Nov 05 2015 20:08
ok
I’ll create some unit tests for this
Blake Niemyjski
@niemyjski
Nov 05 2015 21:54
Can reproduce with a unit test
Modesto San Juan
@modesto
Nov 05 2015 21:58
Can I help in any way?
Blake Niemyjski
@niemyjski
Nov 05 2015 21:58
If I committed a broken test would you be willing to debug it and submit a pull request?
or do you want to pair on this with me?
Modesto San Juan
@modesto
Nov 05 2015 22:00
I think I can try to solve but I'm not sure if I know enough the code.
If its related with LuceneQueryParser it could be hard, but I do not lose anything by trying ;)
Blake Niemyjski
@niemyjski
Nov 05 2015 22:02
:)
Modesto San Juan
@modesto
Nov 05 2015 22:02
Rigth now I could not do pair programming because my children are sleeping in the next room :(
Blake Niemyjski
@niemyjski
Nov 05 2015 22:03
hehe how old are they?
Modesto San Juan
@modesto
Nov 05 2015 22:03
5 and 10
Blake Niemyjski
@niemyjski
Nov 05 2015 22:03
I just committed tests. I’d comment out the ones that pass to make it easier to debug
Modesto San Juan
@modesto
Nov 05 2015 22:16
ok, I'll give it a try!
IsNumeric return false if term == ">30"
Blake Niemyjski
@niemyjski
Nov 05 2015 22:51
yeah
I think I have it mostly figured out but found a few cases we fail on :(
I had to trim that
Modesto San Juan
@modesto
Nov 05 2015 22:53
what should I do ? I can modify IsNumeric() to verify if first chars are < > or = but I'm not sure if it's the best solution
Blake Niemyjski
@niemyjski
Nov 05 2015 22:54
well
the group wasn’t processing the data attribute
the data field
Modesto San Juan
@modesto
Nov 05 2015 22:55
perhaps I could remove this chars before the call to IsNumeric()
the real problem was that this if: if (node.Field != null && node.Field.Field.StartsWith("data.")) must be added to the GroupNode overload too
Blake Niemyjski
@niemyjski
Nov 05 2015 22:56
yeah
and then figuring out the best term node for a group
found 6 cases we don’t handle properly
Modesto San Juan
@modesto
Nov 05 2015 22:58
in the current failing test, we can get a green choosing the left node for the group
Blake Niemyjski
@niemyjski
Nov 05 2015 23:00
yeah
but there is cases that I just found here
left node might be a *
Modesto San Juan
@modesto
Nov 05 2015 23:02
mmm, is left node is a *, how can we calculate term data type?
what kind of query will result in such scenario ?
Blake Niemyjski
@niemyjski
Nov 05 2015 23:05
node
can be date or number
also date one can contain crazy crap like
 [InlineData("data.date:[now/d-4d TO now/d+1d}", "idx.date-d:[now/d-4d TO now/d+1d}", true, true)]
    [InlineData("data.date:[2012-01-01 TO 2012-12-31]", "idx.date-d:[2012-01-01 TO 2012-12-31]", true, true)]
    [InlineData("data.date:[* TO 2012-12-31]", "idx.date-d:[* TO 2012-12-31]", true, true)]
    [InlineData("data.date:[2012-01-01 TO *]", "idx.date-d:[2012-01-01 TO *]", true, true)]
Modesto San Juan
@modesto
Nov 05 2015 23:05
ok, i see.
Blake Niemyjski
@niemyjski
Nov 05 2015 23:06
eh
this kind of stinks lol
Modesto San Juan
@modesto
Nov 05 2015 23:07
mm, Can DateTime.Parse() parse "now/d-4d" ?
:)
Blake Niemyjski
@niemyjski
Nov 05 2015 23:11
nope
Modesto San Juan
@modesto
Nov 05 2015 23:16
I think whe can resolve it step by step and resolve the these date time cases to the end
do you have any other samples to add to the tests ?
Blake Niemyjski
@niemyjski
Nov 05 2015 23:17
yeah
I went through that doc and added them
but I know there is more date range ones you can do
just not seeing them
wouuld you mind looking for date ranges you can do via query string
so far I have theese
  //[InlineData("data.date:>now", "idx.date-d:>now", true, true)]
    [InlineData("data.date:[now/d-4d TO now/d+1d}", "idx.date-d:[now/d-4d TO now/d+1d}", true, true)]
    //[InlineData("data.date:[2012-01-01 TO 2012-12-31]", "idx.date-d:[2012-01-01 TO 2012-12-31]", true, true)]
    //[InlineData("data.date:[* TO 2012-12-31]", "idx.date-d:[* TO 2012-12-31]", true, true)]
    //[InlineData("data.date:[2012-01-01 TO *]", "idx.date-d:[2012-01-01 TO *]", true, true)]
    //[InlineData("(data.date:[now/d-4d TO now/d+1d})", "(idx.date-d:[now/d-4d TO now/d+1d})", true, true)]
Modesto San Juan
@modesto
Nov 05 2015 23:19
is this a typo? [now/d-4d TO now/d+1d}
begin [ and ends }
Blake Niemyjski
@niemyjski
Nov 05 2015 23:20
nope
you can have {} or [] or a combo
Modesto San Juan
@modesto
Nov 05 2015 23:21
ok, I remember! it's for include or not include the value
for now I'll solve some cases , we are not going to resolve all in one try ;)
If you commit the cases you've seen so far I can try to resolve during the weekend.
Modesto San Juan
@modesto
Nov 05 2015 23:26
Luckly I don't need to touch Angular code to resolve this bug, because I've no idea abour Angular :D
Blake Niemyjski
@niemyjski
Nov 05 2015 23:30
omg
@modesto Can you please review this commit
lets pray I didn’t break anything
Modesto San Juan
@modesto
Nov 05 2015 23:31
mmm, what commit ?
ok, I see
Blake Niemyjski
@niemyjski
Nov 05 2015 23:32
exceptionless/Exceptionless@394fa23
If you find any cases that I missed can you please send a pull request for the unit tests
Modesto San Juan
@modesto
Nov 05 2015 23:32
ok
Blake Niemyjski
@niemyjski
Nov 05 2015 23:35
How critical is this bug for you?
I’m just asking cause your self hosted so I’d have to push a new release or I could send you a zip
wouldn’t be so bad to push a new release just wish there was more things that were fixed.
I’m really happy you found this issue… I’m glad it’s fixed as I’m guessing others are running into it.
Modesto San Juan
@modesto
Nov 05 2015 23:37
I can compile it and deploy in my server.
Blake Niemyjski
@niemyjski
Nov 05 2015 23:37
ok
I can send you an artifacts zip too if you’d like
Modesto San Juan
@modesto
Nov 05 2015 23:38
Is crital because we are going to store performance events and we need to filter by durationinmilliseconds range
mmm, that could be great !
oh, sorry for my english. I'm very used to read, but little to write :S
regarding this issue: exceptionless/Exceptionless.UI#38
I have change and test the backend side, but I'm having some problems with angular because of validations. Perhaps some day could you help me to resolve de frontent side of this issue ?
Blake Niemyjski
@niemyjski
Nov 05 2015 23:43
It’s all good :)
yeah no problem!
want to submit a pull request for the server side.
and I’ll take a look and test client side too
Modesto San Juan
@modesto
Nov 05 2015 23:45
ok, thanks!
Blake Niemyjski
@niemyjski
Nov 05 2015 23:46
this one unit test keeps failing on the ci
but can never reproduce it locally :(
Modesto San Juan
@modesto
Nov 05 2015 23:48
are you talking about EnsureSingleRegressionAsync ?
Blake Niemyjski
@niemyjski
Nov 05 2015 23:50
yeah