These are chat archives for nightscout/intend-to-bolus

28th
Nov 2015
Jason Calabrese
@jasoncalabrese
Nov 28 2015 02:50
@bewest is there anything from the pump history that we can use as an id?
running into issues with duplicates, and want to try to prevent it server side in addition to the filtering we do on the pi
Chris Oattes
@cjo20
Nov 28 2015 03:07
jasoncalabrese: duplicates of what?
Jason Calabrese
@jasoncalabrese
Nov 28 2015 03:54
Duplicate treatments going into NS, currently on the pi we fetch the last treatment posted by openaps, then send everything after that point
It's strung together bash scripts, and sometimes with bad connectivity something goes wrong getting that date, and we insert everything again
I tried a couple things to fix it, but it gets messy quick and still want working correctly
Feels like we need something higher level to do the syncing
Scott Leibrand
@scottleibrand
Nov 28 2015 03:58
In the mean time we could also add Ben's dupe removal tool to cron... Or use that same logic...
Jason Calabrese
@jasoncalabrese
Nov 28 2015 03:59
A quick fix would be to upset using some set of key fields
upsert
But should be able to create a signature on the pi
Ben West
@bewest
Nov 28 2015 04:17
type and created_at seem good
Matthias Granberry
@mgranberry
Nov 28 2015 04:21
syncing data is surprisingly hard to get right.
Chris Oattes
@cjo20
Nov 28 2015 04:21
jasoncalabrese: you're stripping _tell?
(if it's even still there)
Jason Calabrese
@jasoncalabrese
Nov 28 2015 04:23
?
Chris Oattes
@cjo20
Nov 28 2015 04:23
in the data from medtronic there used to be an _tell field, which was causing me to have duplicate data
Peter Miller
@ochenmiller
Nov 28 2015 04:25
I've been using date as the key, and always upserting into mongo.
haven't had any trouble with dupes
Chris Oattes
@cjo20
Nov 28 2015 04:29
cjo20/openaps-js@743adc8 that's what I had with my original ns-upload-treaments.sh
_tell seemed to vary depending on the records position in the stream from the pump, rather than the contents of the record. I'd end up with two identical records other than the _tell.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 04:30
Oh, hadn't noticed that field before
Chris Oattes
@cjo20
Nov 28 2015 04:31
but that was a month ago, and before the move to oref0, so it might be out-of-date information. Worth a look though.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 04:31
Think eventType and created_at should work for an upsert
Needs to work for manual careportal entries too, and you may want to enter multiple at a single point in the past
Chris Oattes
@cjo20
Nov 28 2015 04:34
I'd check if it's the _tell issue first
Jason Calabrese
@jasoncalabrese
Nov 28 2015 04:50
The POST to /treatments is just doing a simple insert, but we do an upsert for entries, so no reason not to do it for treatments
Chris Oattes
@cjo20
Nov 28 2015 05:06
Ok. As a seperate issue, meaningless fields probably shouldn't be sent to the server.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 05:09
Agree
Pete Schwamb
@ps2
Nov 28 2015 05:48
Would love to have de-duping on treatments. Finally got some treatments uploading via RL just a few minutes ago, and had to do a double-take at what it thought IOB was. Turns out I had uploaded the same treatments quite a few times. :)
Matthias Granberry
@mgranberry
Nov 28 2015 05:49
@ps2 I'm writing the upload part of a Tandem t:slim uploader right now and I've been putting it off because I didn't want to read in everything to do a smart sync.
Chris Oattes
@cjo20
Nov 28 2015 05:50
@ps2 any chance you could find two copies of a record and see what the difference is between them?
Pete Schwamb
@ps2
Nov 28 2015 05:51
In my case, there were no differences. I had just processed the same page of history data several times.
Well, I assume the _id field was different.
Chris Oattes
@cjo20
Nov 28 2015 05:52
any chance you could paste a pair?
Pete Schwamb
@ps2
Nov 28 2015 05:52
Sure, I can recreate it. 1 sec
Chris Oattes
@cjo20
Nov 28 2015 05:59
hmm, I thought the NS API stripped duplicates
for treatments as well as entries
Jason Calabrese
@jasoncalabrese
Nov 28 2015 06:14
The _id gets added when inserted to mongo, I'll add an upsert based on eventType and created_at
Pete Schwamb
@ps2
Nov 28 2015 06:15
Awesome!
Jason Calabrese
@jasoncalabrese
Nov 28 2015 06:44
a usb extension cable was waiting for me at home, see much better carelink reception with it than with the stick in the pi directly
wish I had that last week
Scott Leibrand
@scottleibrand
Nov 28 2015 06:46
:+1:
Peter Miller
@ochenmiller
Nov 28 2015 19:21
Hank Jr fits perfectly in those reeeeeeaaaaally big pockets of cargo pants. We are soooooo close to pocket sized portability. I'm really excited to lay my hands on a RPi zero. Our local MicroCenter sold out in about 30 minutes I heard. Of course it's about 5 blocks from MIT, so go figure.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 19:28
wish it had built in wifi and ble
but I guess with a small hub it would still be smaller than the 2
Peter Miller
@ochenmiller
Nov 28 2015 19:30
Yeah, Edison hardware, and RPi implementation would be perfect.
RPi zero with a wixel and carelink stick could be pretty compact though. Still thinking about using the wixel to transmit status packets for xdrip to pick up. Then we could avoid the overhead of a hub, wifi, and ble on a RPi Zero rig.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 19:33
with the improved range from the usb cable I'm thinking I might be able to put it in his camelbak, the rpi itself isn't the problem, battery is the main issue
Peter Miller
@ochenmiller
Nov 28 2015 19:34
We're getting really good performance out of a 10,000 mAH Anker battery.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 19:34
I have a few Ankers
but still heavy/big
Peter Miller
@ochenmiller
Nov 28 2015 19:35
I haven't tried a putting the Carelink stick on a cable, in fact I went the opposite and put it inside the RPi case. How big an improvement are you seeing putting the Carelink on the USB cable?
That is definitely the downside. RPi zero pulls about half the current of the regular RPi I hear.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 19:36
an extra few feet is seems
but it did a low temp when he was in the other room
Peter Miller
@ochenmiller
Nov 28 2015 19:37
That's pretty significant. I might try that combined with the antenna mod.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 19:37
might have just been luck, but don't think it would have happened if plugged directly to the pi
it's on a 1ft cable now
was thinking if I get a good case I might do a test run
Peter Miller
@ochenmiller
Nov 28 2015 19:38
Worth some bench testing. Our success - or lack thereof - with the Carelink stick seems to be entirely related to propagation.
Jason Calabrese
@jasoncalabrese
Nov 28 2015 19:39
but during the day with carbs and bolus snoozing it doesn't help that much
but with meal assist that could change