These are chat archives for nightscout/ios

11th
Aug 2015
jwedding
@jwedding
Aug 11 2015 14:03
Hi, @kenstack! Dig the new stuff you guys have put out!
kenstack
@kenstack
Aug 11 2015 14:39
@jwedding thanks ! Getting there
Jason Calabrese
@jasoncalabrese
Aug 11 2015 15:39
AR2 on raw was producing a lot of false alarms for me so I stopped using it, but we got an adult version for my son
jwedding
@jwedding
Aug 11 2015 15:42
@someoneAnyone How hard would it be to get a CarePortal panel added to each site settings?
kenstack
@kenstack
Aug 11 2015 16:54
I'm doing analysis on ar2 right now as its implemented - I'm getting strange results for both the predictions and the triggers - I'll send you some more stuff - I may be doing something wrong though
jwedding
@jwedding
Aug 11 2015 17:29
AR2? I must have missed something...
kenstack
@kenstack
Aug 11 2015 17:36
The predictor
jwedding
@jwedding
Aug 11 2015 17:37
Ah. Gotcha. Sorry, saw this in iOS thought I'd missed some Apple R_ 2 abbreviation.
kenstack
@kenstack
Aug 11 2015 17:40
@jasoncalabrese is there an easy way for me to get the actuals that ns is calculating ie given 2 points what is the prediction at x min later and the loss ? I tried to find the prediction that creates the alarms in chrome tools from my site so I could log them but I can't find it in client.js .... Sorry for the noob question
Jason Calabrese
@jasoncalabrese
Aug 11 2015 17:42
don't think there's an endpoint that exposes that, but thats the type of thing I want on that /summary endpoint
kenstack
@kenstack
Aug 11 2015 18:04
@jasoncalabrese yes no worries - I was going to put a breakpoint in and then just look at the values - Im struggling to find it though
I can find the cone calculation - not the alarm predictor
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:11
there's some debugging for alarms, that includes the predicted values
really want that summary endpoint
and to store notifications/alarms in mongo, and tons of other things
kenstack
@kenstack
Aug 11 2015 18:13
OK yes I see it in websocket.js - so that is not in 0.7.0?
yeah alarm history would be awesime
awesome :)
jwedding
@jwedding
Aug 11 2015 18:16
Humor me, what's the use case on Alarm History?
kenstack
@kenstack
Aug 11 2015 18:16
if alarming at 8pm every tuesday perhaps your basal / bolus is wrong
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:16
and build confidence in the system, see when alarms are acked, by who
kenstack
@kenstack
Aug 11 2015 18:17
its also critical for open aps - alarm histories can show us for example how often we got false alarms, optimizing predictor etc
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:17
also will enable new types of alarms, that are hard to do now, like low battery, and stale data
jwedding
@jwedding
Aug 11 2015 18:17
Gotcha. That makes sense. Just had to ask to learn.
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:18
need to be able to preserve state between server restarts too
kenstack
@kenstack
Aug 11 2015 18:34
@jasoncalabrese so again sorry for noob question but I see the code in websocket.js in my repo but when I search through sources in chrome tools on my website to put in a break or a watch I don't see websocket.js - is it not being used ?
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:35
almost everything has been moved out of websocket.js
it was a monster before
code in dev is totally different than master
kenstack
@kenstack
Aug 11 2015 18:37
Ahh ok - so I should deploy dev if I want to do this ... Got it
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:37
Screen Shot 2015-08-11 at 11.37.02 AM.png
dev vs master
could almost be considered a rewrite
kenstack
@kenstack
Aug 11 2015 18:40
Cool - will try it tonight
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:40
I really like the new pushover alarms, but I know you had a lot of custom stuff
kenstack
@kenstack
Aug 11 2015 18:47
will check it out!
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:48
checkout the readme on dev too, it should have most of the info for the new plugins
kenstack
@kenstack
Aug 11 2015 18:48
the key is the predictor accuracy and avoidance of false alarms. AS soon as I confirm Im using the AR2 from lane correctly Ill send you some graphs of real data vs predicted vs loss function value.
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:49
using raw for AR2, helped catch some lows faster, but was triggering false alarms for highs
don't have a great example, but in cases like this
Screen Shot 2015-08-11 at 11.50.42 AM.png
raw picks up the change faster, but that would cause AR2 to forecast wrong
Screen Shot 2015-08-11 at 11.51.25 AM.png
with the retro mode you see the AR2 forecast using the sgv values lines up nice
Scott Leibrand
@scottleibrand
Aug 11 2015 18:53
Is that using 5m or 10m lookback?
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:53
5m
not using the 10m now, seems that it was only needed for the old dex algorithm
Scott Leibrand
@scottleibrand
Aug 11 2015 18:54
might also be useful for raw ar2 (or even 15m)
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:55
but maybe 10m for raw AR2 would be good
yeah
kenstack
@kenstack
Aug 11 2015 18:55
Im not sure its wrong, the question is which BG at the moment is right and which slope is right. Scott's point on 5 or 10 is a huge debate its why Im trying to do some compares and calc errors etc - accurate slope is the key
but which slope is more accurate? who knows - depends right?
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:56
the smoothed dex values do seem to work well for forecasting
and they new AR2 point is usually very close to raw
kenstack
@kenstack
Aug 11 2015 18:56
yes its likely their derivative smoothing is outweighing some of the noise of raw - but - the peds algo is so slow you may miss a drop for 15 min
so in dev you get a prediction and bounds?
Scott Leibrand
@scottleibrand
Aug 11 2015 18:57
yeah, dex is smoothing raw, so if we want to use raw to do better, we need some sort of smoothing too.
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:57
yeah, we were able to get a prescription for the adult verion
Scott Leibrand
@scottleibrand
Aug 11 2015 18:57
Not sure the ideal algorithm, but 10-15m lookback is one easy way to smooth.
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:58
the clinic is now only prescribing the adult version
kenstack
@kenstack
Aug 11 2015 18:58
totally agree Scott! Im playing with 3 point derivative - I need to make sure though my ar2 is calcing right first
nice on your clinic - mine doesnt even like CGM:)
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:58
with the new plugins forecasts are generic
so we can plug in a new forecast
kenstack
@kenstack
Aug 11 2015 18:58
cool
Jason Calabrese
@jasoncalabrese
Aug 11 2015 18:59
I'd like to use multiple at once, show them all on the chart
kenstack
@kenstack
Aug 11 2015 18:59
will try to deploy dev tonight to confirm
blob
Jason Calabrese
@jasoncalabrese
Aug 11 2015 19:03
never seen the loss graphed like that, interesting
kenstack
@kenstack
Aug 11 2015 19:03
im doing live compares like this - but my results dont look great. This is what I think is the right ar2 from NS versus actual BG (green), predicton (blue) and the loss function... the loss function as Im calcing it basically matches low and high thresholds - because I think the algo lags so badly. That said, I may be doing the calcs wrong which is why I want to confirm my predictor doesnt have a typo
trigger is 0.05 - look at the low and high times versus actual BG -may as well use the thresholds - that said, Im likely doing something wrong
Jason Calabrese
@jasoncalabrese
Aug 11 2015 19:06
for loss you're including/weighting all the forecasted points
kenstack
@kenstack
Aug 11 2015 19:06
Im literally using the code from Ross thats in NS
Jason Calabrese
@jasoncalabrese
Aug 11 2015 19:07
hmm, could be that AR2 isn't as fast as I thought
kenstack
@kenstack
Aug 11 2015 19:07
for a given point in time I compute predictions for 30 min (I happened to graph the 15 min one in the plot) and calc the loss function for each point as NS does - it should match the alarms - thats why I want to log NS's calcs to the console I may have a bug
its really laggy
Jason Calabrese
@jasoncalabrese
Aug 11 2015 19:08
or maybe those default thresholds aren't great
kenstack
@kenstack
Aug 11 2015 19:08
but remember those contrived data plots I sent you? Matches those. Again may be a bug or me not understanding something
let me confirm it isnt me before we jump to any conclusions
Jason Calabrese
@jasoncalabrese
Aug 11 2015 19:09
I thought the issue there was forecasting on the forecasted points
I didn't remove that round yet either, and Ross said it wasn't needed, so that may help some
kenstack
@kenstack
Aug 11 2015 19:10
I want to write some javascript/html code that we call can use to compare alarm triggers to various predictors - right now Im kind of flying bind - I use simpler stuff in my own code but its prob too agressive - hence I want to see what Lanes AR2 does
I did
remove the round
right - all predictors - unless you want to only look one timestep ahead - are predicting on forecasted points - thats why the error builds a lot
let me get dev installed so I can get the same graph you showed and I can compare apples to apples
for me and Ryan, Im mostly interested in downside prediction - for rises Im more OK chasing a high bg and not overbolusing
kenstack
@kenstack
Aug 11 2015 19:39
Ive got dev installed - no prediction just the cone displays
Jason Calabrese
@jasoncalabrese
Aug 11 2015 19:42
the cone is AR2, you can make it a single line (AR2_CONE_FACTOR=0) or make it narrower (AR2_CONE_FACTOR=1), default is 2
I like using 1
kenstack
@kenstack
Aug 11 2015 19:43
AHHH ok - you are using the midpoint of the cone? I was using the AR2 calc from websocket.js
let me add that to a version of my code and compare the 2
Jason Calabrese
@jasoncalabrese
Aug 11 2015 19:46
I think I did even more refactoring of ar2 since you last saw it...
uses _.reduce now, so that the cone and normal forecast that's used for alarms share the same code
kenstack
@kenstack
Aug 11 2015 19:49
you have been a busy guy :)
thanks!
Ill check this out tonight and try to compare it
kenstack
@kenstack
Aug 11 2015 20:23
quick check on a couple cases are matching exactly on predicted BGs - gotta do some real work will get back to this tonight
kenstack
@kenstack
Aug 11 2015 20:58
bgs are matching exactly to results.predicted[0] thru results.predicted[5] example bgold=125, bgnew=128 (five min interval), @ 30 min bgpredict=136
I think these charts still hold now that I think my ar2 code matches - I think they illuminate the issue I am seeing
blob
this if course assumes the midpoint of the cone matches results.predicted, which I assume it does? I couldnt figure out how to enable the extended settings.....
Jason Calabrese
@jasoncalabrese
Aug 11 2015 21:03
where are you running your site? it's just an other setting like mongo or enable
kenstack
@kenstack
Aug 11 2015 21:03
on azure
whats the key word?
or is it the key word?
Jason Calabrese
@jasoncalabrese
Aug 11 2015 21:04
add the AR2_CONE_FACTOR setting with the value 0
kenstack
@kenstack
Aug 11 2015 21:04
ahh:) sorry man
Jason Calabrese
@jasoncalabrese
Aug 11 2015 21:04
think it's in site settings
still have never used azure
Ben West
@bewest
Aug 11 2015 21:05
configure tab -> App Settings
Paul Andrel
@stavlor
Aug 11 2015 21:05
yep
kenstack
@kenstack
Aug 11 2015 21:15
got it now sorry guys
Paul Andrel
@stavlor
Aug 11 2015 21:20
np
kenstack
@kenstack
Aug 11 2015 21:22
matches the single line in a couple of tests - thanks guys
check out those contrived data examples above - really helps you get a feel for the based on an initial velocity (1 mg/dl/min in the cases I ran) what the results from the AR2 are going to be. I think the biases you see (ie at a high bg and rising BG or a low bg and falling) we predict a VERY small change in BG - this is likely because the data that was used to fit the model were from actuals - if you are high you have likely treated with insulin - if you are low you have likely treated with carbs - therefore the AR2 gives almost no change - this is true in real life for many cases of course - BUT - Im not sure this is what we want for alarms..... kind of scary - if you are 80 and dropping hard it gives you a very small prediction for the drop in 15 or 30 min....
now that I feel better that my ar2 doesnt have a bug Ill try to create some code so you can compare your own actuals from the pebble end point or something and see when alarms would have been triggered....
Jason Calabrese
@jasoncalabrese
Aug 11 2015 21:35
ar2 does seem to work best for fast changes, when the slope is much bigger, with the 1 point drop the central tendency has the biggest effect
so a drop from 120 to 90, would trigger an alarm, before a threshold of 80, but 80 to 79 wouldn't
kenstack
@kenstack
Aug 11 2015 22:01
1mg/Dl/min -so 5 per 5 min