@dmetzger57 So ran the patched code with a VIM Broker as well, and I saw improvements on each provider size: Small - ~7%, Medium: ~26%, Large ~46%, XLarge 75%
On the X-Large the non-patched code took 3585s total on the best run (4 runs) and the best run with the patched code was 898s
@akrzos While not optimal, this is more acceptable.
So thats only Initial Refreshes too so not sure if there is a subsequent improvement in targeted refreshes etc
@dmetzger57 Thats VIMBroker Total timings
Also note I don't track the amount of time it took to "seed" the VIMBroker
the post initial refresh improvement is proportional to the number of VMs being linked, so the impact is tied to the number VMs that are new / added
thats what i thought so I didn't run any targeted tests, but I can certainly run those as well
@dmetzger57 I was looking at sam saffron debugging memory leaks and reminded of post by jnunemaker - would love to get this data into statsd/graphite/influxdb/whatever
@matthewd your bulk connect idea is a nice win for the XLarge environment
seems odd that @akrzos would have to parse logs to get data that was already structured before we put into the logs in the first placec
I assume you've already read old post measure everything that kicked off the statsd kraze
no, but it's on my reading list now :-)
So to track memory leaks, we would need a long term "stress" testbed
etsy goes quite deep and kicked off a bunch of tools (in the hundereds)
well, not sure if we need to track memory leaks
scale environment would be actually perfect for this
just want to start tracking SOMETHING
+1 statsd all the things
want to see how many queries, how much data throughput, ... we do on the same test runs over time
disclaimer I say statsd but mean statsd / graphite / flavor of the month (which is influxdb for me right now)
but the idea is to make metric colletion so easy, that you start measuring everything. and then you can detect regressions, and when you performance tune, you can actually see things getting better
ala ruby bench