Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Arthur Madec-Prévost
    @arthurmadecprevost
    Hello everyone! Good job for your app. I'm currently testing it and it's such a great thing for a better IT. Do you plan to extend it to other Hypervisors ? Such as VMware, Citrix, Hyper-V ?
    bpetit
    @bpetit

    Hi ! Thanks ! I intend to work on VMWare hypervisors support at some point, but there are several things first in the roadmap. Any help on this would be highly appreciated ! :)

    For Hyper-V first thing would be to add windows support : hubblo-org/scaphandre#74
    Then why not.

    Actually, explanation of supporting only qemu/kvm is not rooted in some deep technical explanation. We just only tested in this context. The data coming from the hypervisor is exposed in the virtual machine through a read only volume, which is not that complicated and should be replicable in vmware (maybe) or other linux based hypervisors.
    Arthur Madec-Prévost
    @arthurmadecprevost
    Thank you for the answer! :D
    Pierre Rust
    @PierreRust
    Hello, I'm playing again with the json exporter
    I have a small question : the consumption in the output is the energy consumed since the previous measurement ?
    in joules ?
    bpetit
    @bpetit
    it is microwatts, it's the power consumption calculated from the energy consumption difference (microjoules) between two measurements and the time step.
    Pierre Rust
    @PierreRust
    ok, thanks for your answer !
    Pierre Rust
    @PierreRust
    I'm currently testing the PR for the json exporter (with timestamp) and there's a few thing I find strange
    the sum of the consumptions for both sockets is not equal to the host consumption
    bpetit
    @bpetit
    which PR ?
    Pierre Rust
    @PierreRust
    yes
    bpetit
    @bpetit
    It's really WIP, it's kind of idle. I wanted to have your thoughts on how it may look like :)
    Could you send me the json result with the bug you mention ?
    so that I look at it
    Pierre Rust
    @PierreRust
    I like it so far :) I currently trying to use it to see if anything important is missing
    here is an example of json :
    {
    "host": {
    "consumption": 25351920,
    "timestamp": 1620112643.986319
    },
    ...
    "sockets": [
    {
    "id": 0,
    "consumption": 15789177,
    "domains": [
    {
    "name": "core",
    "consumption": 1366266
    }
    ]
    },
    {
    "id": 1,
    "consumption": 9785400,
    "domains": [
    {
    "name": "core",
    "consumption": 958565
    }
    ]
    }
    ]
    }
    bpetit
    @bpetit
    I'd like to have your thoughts on this comment too: https://github.com/hubblo-org/scaphandre/issues/75#issuecomment-816623016
    Pierre Rust
    @PierreRust
    that's the way I'm using it right now, looks fine :) I'd rather have a counter instead of a consumption (or both) but that's a detail
    bpetit
    @bpetit
    we could provide both
    regarding the sum, I have to look deeper in the code but I think we dont collect the socket-level consumption exactly at the same time as the host consumption (this is done in the same function but we don't synchronize). So I guess we should provide the timestamp for both kind of measurment
    Pierre Rust
    @PierreRust
    BTW, it seems that the processes' consumption is computed is each exporter, is that not a bit dangerous ? different exporter might expose differents values and a fix in an exporter may not be propagated to others
    bpetit
    @bpetit
    This is not present is that PR as it is quite old, but we have started to create a common interface for exporters at a higher level, to have the same behaviors
    Pierre Rust
    @PierreRust
    or may I just don't understand the code !
    in json.rs :
    consumption: ((*value as f32
                            / (host_time * procfs::ticks_per_second().unwrap() as f32))
                            * host_power as f32),
    20 replies
    bpetit
    @bpetit
    you can see an example in main, in riemann or prometheus. Here it is: https://github.com/hubblo-org/scaphandre/blob/main/src/exporters/riemann.rs#L152
    the functions get_*_metrics
    Pierre Rust
    @PierreRust
    ok, that's a change that is not yet merged in the json exporter PR, is that right ?
    bpetit
    @bpetit
    it is merged in main, but this branch is not up to date yes
    Pierre Rust
    @PierreRust
    Hello, I'm looking right now at the code for the MetricGenerator
    I would like to generate process metrics in the json exporter the same way it is done in other exporters, i.e. using the MetricGenerator, but the Metric struct does not include the timestamp.
    And when looking at the Topology, is seems that the timestamp for the host consumption is set to the point when record are generated (refresh_record) which is not the time when the measurement are actually read , is there a reason for that ?
    bpetit
    @bpetit

    Hi, great !

    I didn't remember that. We should move it closer to the actual measurement. Regarding the Metric struct, it's been made after the json/timestamp PR and we didn't catch up yet.
    Feel free to propose changes for both topics if you have the time.

    So no good reason for that no.
    Pierre Rust
    @PierreRust
    ok, I just added some comments on the PR
    BTW, do you know how I can push "cleanly" to the current PR #75 ?
    3 replies
    Pierre Rust
    @PierreRust
    Hi all
    I'm still trying to see how we could factorize metric generation into a single place
    To make sure all exporters actually export the same data. Currently the MetricGenerator is supposed to to that but we're not really there yet : the gen_all_metrics is only used by the prometheus exporter, sdout json and riemann exporters all implement their own logic for process consumption (which is for me the most important metric)
    Pierre Rust
    @PierreRust
    and actually, process metrics are computer in Topology ...
    well, I'm not sure how to make that cleaner (and safer, especially !), any ideas ?
    bpetit
    @bpetit

    Hi,

    That's a good point, we needed some specific behavior for the riemann exporter so we didn't used the generic function. We should definitely find a cleaner way to do that.

    The power consumption calculation for each process could be easily moved to MetricGenerator though, as it only requires to have access to both the topology and ProcessTracker.

    I need to have a closer look to add some thoughts to the discussion about the gen_process_metrics function.
    Pierre Rust
    @PierreRust
    Hi. when looking into that, there are maybe some other refactoring needed.
    for example to implement a msr or perf_event rapl reading, these method are currently in the CPUSocket, etc. implementation, which make it hard to provide several alternatives
    René Ribaud
    @uggla
    Hello, by the way I will probably introduce a new sensor for GPU. So I may change a couple of stuff around that in order to allow 2 kind of sensors.
    Maybe I should wait you finished with #75 ? Thanks to advice on this.
    René Ribaud
    @uggla
    @bpetit , I missed that : That's a good point, we needed some specific behavior for the riemann exporter so we didn't used the generic function. We should definitely find a cleaner way to do that.