Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Becca P.
    @bpownow_twitter
    @arnikola i DMed
    Daniel
    @DanielQujun
    HI, i'd like to know how many metrics stored in my m3dbnode cluster , is there any way to approach this ?
    Matej Zerovnik
    @matejzero
    Hey guys... Not strictly m3db technical question. Some time ago I had it in my mind that you had info on github/webpage where you got your logo
    Does anyone remember the site?
    Benjamin Raskin
    @benraskin92
    hey I think ultimately we used 99designs
    Matej Zerovnik
    @matejzero
    yes, that was it:)
    xmcqueen
    @xmcqueen
    I like @DanielQujun's simple question. I'm a newbie and I don't know the answer
    martin-mao
    @martin-mao
    Great question @DanielQujun - so the best way is to go to the M3DB Node Details Dashboard (https://grafana.com/grafana/dashboards/8126) and look at the Ticking panel (panel 27 I think)
    now, it's not quite exactly how many unique series are stored in the m3db node cluster completely because of the way data is stored in M3DB (time series are stored inside time based blocks that you configure). So the Ticking graph shows you how many unique series for the most recent block we've persisted. We don't track how many series overall since that changes all the time as series gets introduced and expired at the end of retention and generally, the number of active series is what determines resource usage, not overall number of series
    we'll update our FAQ section of our docs to include this moving forward
    Daniel
    @DanielQujun
    thanks! @martin-mao very helpful answer! M3db is amazing project.
    Daniel
    @DanielQujun
    If you don't mind ,another question ~ I found Tagged RPC panel in grafana , can i do the simple formula WriteTaggedSuccess/Second multiply by 60 multiply by 60 multiply by 24 , then get the approximately number of daily metrics ingested?
    martin-mao
    @martin-mao
    no because 1) WriteTaggedSuccess is the number of batches we wrote, not the number of metrics
    the number of metrics per batch is variable
    you can do something like commitLogWrites per second and apply a moving average to that, but you can only use that to calculate the total number of metric datapoints ingested per second
    not the number of unique time series
    Daniel
    @DanielQujun
    got it, thank you!
    spd-code
    @spd-code
    Is there any http batch post endpoint for metric ingest ?
    martin-mao
    @martin-mao
    @spd-code there is not, but if you use the m3db go client, it will batch for you or you can go via prom remote write and the coordinator (which just uses the m3db go client) will also batch
    spd-code
    @spd-code
    Sure, will try that , is there any plan for a post
    batch endpoint though, that will be useful I think ?
    arnikola
    @arnikola

    If you’re interested, you could submit a PR adding a batched endpoint; there’s some prior work for an endpoint that writes a single JSON datapoint here, but it’s not super well fleshed out or performant if you want to look at it

    https://github.com/m3db/m3/blob/master/src/query/api/v1/handler/json/write.go

    There are no plans for us to add batched write endpoints to query at this time
    Charles Niu
    @charlesni
    hi guys, i find the max of step is 5m when i use query_range, is it correct?
    Charles Niu
    @charlesni
    Isn't the smaller the step in query_range, the more accurate the result of using sum function?
    xmcqueen
    @xmcqueen
    if m3db is storing mostly cluster config/topo in etcd, then where is it storing the actual metrics?
    xmcqueen
    @xmcqueen
    I see something here about persistent storage and commit logs: https://m3db.github.io/m3/m3db/architecture/engine/. So I guess its storing to disk similar to redis
    and fileset files and writepath. I guess I'll read that page more
    arnikola
    @arnikola
    yeah data and index files are written to disk
    they’re kept in memory while your current block is active
    then get flushed to disk
    Charles Niu
    @charlesni
    image.png
    How to understand the "step" in / api / v1 / query_range？
    arnikola
    @arnikola
    It’s the same as https://prometheus.io/docs/prometheus/latest/querying/api/#range-queries
    Charles Niu
    @charlesni
    If I use this API, what do the 10, 5, and 5 in the results mean?
    image.png
    Benjamin Raskin
    @benraskin92
    @charlesni those are just the values of each datapoint
    xmcqueen
    @xmcqueen
    is it expected to have "make" from the HEAD of master to fail badly? I wonder if my local is screwed up somewhere or if HEAD is often broken
    it can't be broken if github is showing its green. I must be hosed
    arnikola
    @arnikola
    Which are you trying to make?
    xmcqueen
    @xmcqueen
    i was doing "make" default target. It seems the individual services build
    i'm really not sure what's its making by default. its still pretty mysterious to me
    arnikola
    @arnikola
    I’m not even sure make default does anything useful haha
    Charles Niu
    @charlesni
    @benraskin92 How do I get the result if I want to calculate the total in 30 seconds?
    Is this API correct? http://localhost:7201/api/v1/query_range?query=sum(login{ip="123"})&start=1576206806&end=1576206836&step=1s
    Charles Niu
    @charlesni
    It returns too many values, which value is the statistical result of the sum function？
    Thanks for your answer
    Benjamin Raskin
    @benraskin92
    hmm what do you mean by calculating the total in 30s? like you want to sum in 30s buckets?
    Charles Niu
    @charlesni
    @benraskin92 yes, 😂
    Benjamin Raskin
    @benraskin92
    sounds like youll want sum_over_time
    https://prometheus.io/docs/prometheus/latest/querying/functions/#aggregation_over_time