Monitoring is just hard :-) Lots of writes, and generally a large active dataset for reading.
Timothy Perrett
@timperrett
myself and @djspiewak have discussed alternitive designs recently for changes to the internals, but its non-trivial.
yep exactly
i’d really like to get a multi-vector model - what we have is probally a little too constrained
Gary Coady
@fiadliel
like the labels prometheus has — it’s powerful but complex to understand, still useful for pivoting across different views on your data.
Timothy Perrett
@timperrett
yeah - im torn on its power. There are cases where i wish we had that, but the simplicity of what we currently do also has its alure.
i like simple solutions
or at least, the minimally powerful thing
Gary Coady
@fiadliel
when it comes to monitoring though, sometimes it’s a badly behaving user (show latency by user), sometimes it’s a bad network switch (show latency by rack), sometimes a machine has a bad NIC, perhaps your new software revision is bad (divide by software version). So it can be useful.
Timothy Perrett
@timperrett
@fiadliel oh im totally with you
Gary Coady
@fiadliel
But last time I used a monitoring system that powerful, one person on our team basically wrote monitoring rules full-time ;-)
Timothy Perrett
@timperrett
@fiadliel essentially the ssytem needs to be powerful enough to do the nessicary, but simple enough to use with a few basic conventions, and the system should absolutly plan for abuse by the devs
so having ways to cap usage or throughput is key
on the basis that the system WILL be abused
Gary Coady
@fiadliel
if you are in charge of pulling the data, the only danger is #time series there, and with labels you get a multiplicative effect. So probably some kind of quota would be useful.
Then there’s also a cost on querying.
Timothy Perrett
@timperrett
yup yup
nice to chat with someone who also appreciates the complexities of monitoring :)
Gary Coady
@fiadliel
I was an SRE with Google at one point ;)
(eh, ops / site reliability engineer)
I have to say, 1 second granularity on latency split across many dimensions, is completely awesome, but very expensive!
I’ll have a think about things anyway; thanks for the chat
Timothy Perrett
@timperrett
np
Guillaume Massé
@MasseGuillaume
hehe I see lot's of action in the package index from this project
@MasseGuillaume I think it is an issue for all oncue projects on bintray, same thing for remotely artifacts. Could someone with credentials please look into this? /cc @timperrett@stew@runarorama
Timothy Perrett
@timperrett
@ahjohannessen hmm - i’ve no idea what would be causing that
the file shows as present
@ahjohannessen are you sure this just doesnt happen in the browser?
it could just be a bintray “feature"
i mean, the actual artifacts do resolve and download
Timothy Perrett
@timperrett
@MasseGuillaume@ahjohannessen bintray did something stupid to our account; i’ve emailed them
Guillaume Massé
@MasseGuillaume
Great let me know when they resolve your issue !
Alex Henning Johannessen
@ahjohannessen
@timperrett
are you sure this just doesnt happen in the browser It also happened on shippable that it could not resolve artifacts
Dan Billings
@danbills
FYI I’m seeing a deadlock on the elastic publisher thread pool, investigating
Dan Billings
@danbills
^ disregard, is upstream of elastic
Timothy Perrett
@timperrett
@ahjohannessen@MasseGuillaume bintray issue is resolved
Alex Henning Johannessen
@ahjohannessen
@timperrett awesome, thanks :+1:
Guillaume Massé
@MasseGuillaume
@timperrett cool thanks !
If you have any special request for the scala index let me know ;-)
this issue caught my attention because there was a lot of failures associated to oncue's Bintray repos
Rob Norris
@tpolecat
hello lads, what do i need to do the simplest kind of single-process instrumentation?
The doc says libraryDependencies += "oncue.funnel" %% "http" % "x.y.z" which doesn't exist.