Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Timothy Perrett
    @timperrett
    @runarorama @stew so i revived this, as some people want the direct-push model, so i didnt see any reason to restrict that from happening. WDYT? oncue/funnel#20
    Mike (stew) O'Connor
    @stew
    :shrug: I dunno, I still don't know what Reimann is. This one seems weird: https://github.com/oncue/funnel/pull/20/files#diff-06764678f4806cde8e5131fa19042f3bR130
    but whatever if people want it, they should probaboly have what they want
    Timothy Perrett
    @timperrett
    @stew yeah, that bit is weird. I cant recall why we did that - i think its just to batch into groups of a 1000 or something. i should probally look into making this more like the elastic module
    literally just revived this from the history
    so i made more progress on getting the quickstart docker compose setup
    need to revise something in the way chemist expects configs but should have it running tomorrow
    its pretty neat though: spins up elastic search, kibana, 2 flasks, 3 targets and a chemist
    Dan Billings
    @danbills
    @runarorama I'm trying to validate via unit test that the Discard command to a Flask will disconnect from a host.
    I can verify that the io.resource "cleanup" doesn't run. If I throw a println in the "step" function of io.resource, it appears to run forever.
    It now occurs to me that I should validate this bug locally so that i can verify it is indeed a bug and not an issue w/ my test :)
    Rúnar
    @runarorama
    that's a good idae
    idea
    Dan Billings
    @danbills
    brb
    Rúnar
    @runarorama
    seems like maybe processMirroringEvents is doing something nefarious
    Dan Billings
    @danbills
    My hunch is attemptRepeatedly
    Rúnar
    @runarorama
    p.append(schedule.kill).attempt(e => Process.eval { Task.delay { maskedError(e); e }})
    append(schedule.kill) specifically
    Dan Billings
    @danbills
    going by documentation that kill command should "giving chance for any cleanup actions to be run"
    Rúnar
    @runarorama
    check if io.resource cleanup actually survives appends
    Dan Billings
    @danbills
    ok
    Rúnar
    @runarorama
    also, is attempt somehow interfering here?
    Daniel Spiewak might have a good idea
    Dan Billings
    @danbills
    I tested attemptrepeatedly with a schedule of two "ticks". It runs forever
    shouldn't the last line just be retries.XX instead of (step ++ retries)
    since they're zipped
    eh it still would only run once
    Dan Billings
    @danbills
    pretty sure I get it now. p.append appends a kill to an endless process
    Dan Billings
    @danbills
    I don't think that append(schedule.kill) is necessary. The zip w/ the schedule should run any cleanup code. Moreover, the schedule in flask's case doesn't need to be cleaned up.
    I'll submit as such
    @runarorama ^
    Rúnar
    @runarorama
    Awesome. Thanks!
    Dan Billings
    @danbills
    @runarorama am still struggling to fix this bug with Discard message -> TCP disconnect. Could you review my test? Am pushing it to my PR now
    Rúnar
    @runarorama
    Hey @danbills, I don't see a commit from today in that PR
    what am I looking for?
    Dan Billings
    @danbills
    @runarorama I just pushed a new file of tests as MonitoringSpec takes 1.5 minutes to run the earlier tests. https://github.com/danbills/funnel/blob/unit_test_key_removal/core/src/test/scala/ProcessMirroringEventsSpec.scala
    Rúnar
    @runarorama
    @danbills ping me on the Google and we can do a hangout
    Gary Coady
    @fiadliel
    Hey, I was wondering if anybody has any pointers on trying to estimate a PDF for service latency from the standard moments? (I’m used to estimating percentiles with bucketed histograms to see a service’s outlier performance, but if the current statistical data can do a good enough job…)
    Timothy Perrett
    @timperrett
    @fiadliel what acronym are you ascribing “PDF” ? im not familiar with it
    Gary Coady
    @fiadliel
    probability distribution function
    Timothy Perrett
    @timperrett
    ah ok
    :)
    @fiadliel im not aware of anything like that, we’ve done some things internally with post-processing the funnel data, but nothing thats public right now.
    Gary Coady
    @fiadliel
    Okay - well other than that, maybe will try to resurrect the Histogram type. It is quite hard to get a good feel for the distribution shape without further work on top of the raw numbers.
    Timothy Perrett
    @timperrett
    @fiadliel yeah that would be cool - i’d welcome that
    @fiadliel im curious, what are you using for monitoring right now?
    Gary Coady
    @fiadliel
    how I’ve seen it done before is either buckets with exponentially increasing latency (1ms, 2, 4, 8, 16, …, MAX) or rounded numbers (1, 2, 5, 10, 20, 50, 100, …, MAX) which are more human-readable. And the obvious nice quality that this can be aggregated across instances. But it’s a lot of extra time series.
    I hate to even say it, we’ve historically mostly used NewRelic, I won’t say any more :-( :-( There is also some monitoring with InfluxDB, and AWS CloudWatch. We’re not using Prometheus, but I’ve looked at it a tiny bit, it’s not as clean as funnel, but then not many services are :-)