These are chat archives for juttle/juttle

6th
Feb 2016
Phil Rzewski
@philrz
Feb 06 2016 17:39
Am I correct that there's no way to create batch boundaries based on rhythm of incoming data rather than time intervals? I've got a use case where my Meraki wireless access points send periodic HTTP POSTs that contain reports of wireless activity, but not on a perfectly predictable schedule like a statsd/collectd or whatever. Each POST basically reflects "a full/current report of reality". So I want my downstream Juttle logic to fire based on "when there has been a change in reality", hence clocked based on the end of each post rather than a static time rhythm. So I'm guessing what I'd be seeking is an option for the HTTP Server Adapter to add those marks after each POST it receives.
Daria Mehra
@dmehra
Feb 06 2016 17:43
Are you asking for each data point to be its own batch? Or batches of every N points?
Phil Rzewski
@philrz
Feb 06 2016 18:02
Batches of points, but not even where N is a predictable number. Sometimes it might report 20 points, sometimes 30.
That's why it seemed like the boundary-setting would need to start in the HTTP Server Adapter, since that's where it first has visibility to the fact that the points all showed up as part of a single POST.
Oleg Seletsky
@go-oleg
Feb 06 2016 21:26
@philrz, can you post an example of one of these POSTs? do all the points in the same POST end up with the same time? if they do, you could send them in through a join that will merge all the fields in all the points of a given time into one point. i’m thinkin you can then use the delta reducer (http://juttle.github.io/juttle/reducers/delta/#delta) or write your own custom one and use in put context to look for changes across POSTs. seeing the data and the kind of change you are trying to track would be helpful.