hubblo-org/scaphandre

General channel to discuss about the scaphandre project (https://github.com/hubblo-org/scaphandre/)

MarwaSlimene

@MarwaSlimene

Hi Guys,
I want to know, is it possible to integrate the Scaphnadre in a pipline CI/CD to measure the energy consumption of an application during the test phase for example?

3 replies

bpetit

@bpetit

Performing some upgrade on the ci/cd + demo machine, don't worry if some jobs fail or you dont see the metrics dashboard ;)

bpetit

@bpetit

Performing new upgrades, ci and dashboard may be unavailable during the afternoon.

René Ribaud

@uggla

Hello @bpetit, reviewed #148.

I guess it would be great to have someone else looking at it, because it is a massive one.

Also just a couple of news regarding OpenStack integration. As you know BBC and power management was a thing at latest summit.

René Ribaud

@uggla

So we will write two specs (one on scaphandre/virtiofs integration, one allowing to stop CPUs (unused)) regarding this topic for the next PTG (October). Then the implementation should go on the AA release cycle. (Note: I'm the owner of the first one and I will probably do the nova implementation.)

René Ribaud

@uggla

And to finish, I looked at #176, that's interesting. Maybe it could be used to add integration tests. To my mind this is something becoming a priority as with new functionalities it will become harder and harder to change the code without regression.

I also had a look at mockall in order to mock sensors and so create exporter tests to ensure changes will not break them. The mockall create looks powerful and that looks possible to do it. But I might need help to better understand how to mock sensors.

bpetit

@bpetit

Hi !

I guess it would be great to have someone else looking at it, because it is a massive one.

yes this would be super nice. If anyone here @/all is interested into making a review as well, that would be great !

So we will write two specs (one on scaphandre/virtiofs integration, one allowing to stop CPUs (unused)) regarding this topic for the next PTG (October). Then the implementation should go on the AA release cycle. (Note: I'm the owner of the first one and I will probably do the nova implementation.)

This is awesome. I'd love to help with the first one then, tell me what you'd need. On the second one, did you see this ?

And to finish, I looked at #176, that's interesting. Maybe it could be used to add integration tests. To my mind this is something becoming a priority as with new functionalities it will become harder and harder to change the code without regression.

#176 is definitely interesting, I'll have a look at it and see how we could use it in the CI tests, just after release 0.6.

I also had a look at mockall in order to mock sensors and so create exporter tests to ensure changes will not break them. The mockall create looks powerful and that looks possible to do it. But I might need help to better understand how to mock sensors.

This sound interesting as well. I'll happily review any suggestion. I also 100% agree on the need of integration tests and even maybe some e2e tests (better than the drafts I've made in #148

bpetit

@bpetit

Hi !

Through Boavizta, we are trying to reinforce international collaboration on ICT environmental impacts evaluation tools methods and data.

We initiated a mailing list where we can debate related topics, announce talks to be performed to the members of the mailing list (next one is in september), issues that could be discussed collectively to better understand and address this complex field.

The group already gathers members of Boavizta, the SDIA, the Green Software Foundation, OurTechLegacy, to name a few.

If you'd like to join, could tell me by DM and send me the email address to be used for your subscription ?

tormath1

@tormath1:matrix.org

[m]

send me the email address to be used for your subscription ?

Hi, subscription is only to the mailing list?

2 replies

Sebastian Werner

@tawalaya

Hi, I don't know if it's the right place. Just let me know if it is not. I want to try to use scaphandre in a KVM cluster. However, I encountered an issue with the qemu setup, and I'm unsure if I did something wrong or if its a bug. It runs on the host and creates the folder per domain as it is supposed to be. However, the folders to be mounted into the VM are all empty, or rather the folder with ...0:0 is empty, and the other only contains a file with a 0 in it. I'm guessing there is supposed to be more. If i mount it in the VM and run with scaphandre --vm after mounting it. I get the following error:

if you are running on a VM, do not forget to use --VM parameter invoking scaphandre at the command line, src/sensors/mod.rs:263:18

.
Could you give me any guidance on how to troubleshoot this?

tormath1

@tormath1:matrix.org

[m]

Hi @tawalaya what's your KVM host OS and the guest OS? The required Linux modules are present?

1 reply

tormath1

@tormath1:matrix.org

[m]

and on the host it's running with scaphandre qemu ?

Sebastian Werner

@tawalaya

yes

tormath1

@tormath1:matrix.org

[m]

I recently wrote this gist: https://gist.github.com/tormath1/140a625178588ae9e5b76d7342012cfc - you can double check.
It's a Gentoo host too.

You can skip the Flatcar part and goes directly to the end

Sebastian Werner

@tawalaya

Hi, i checked it. The kernel config looks right. The mout and VM config also look the same. I starter the qemu exporter with verbose logging and I also see logs. So I figure its something with the shared fs? (I put the logs from the qemu exporter into this pasetbin: https://pastebin.com/3Epj2rQq)

Inside the vm something is mounted cat /var/scaphandre/intel-rapl\:0/energy_uj shows a 0 and /var/scaphandre/intel-rapl\:0\:0/ is empty

Sebastian Werner

@tawalaya

if you have this running somewhere could you let me know whats inside the two folders in /var/lib/libvirt/scaphandre/flatcar-linux1 in your example?

Sebastian Werner

@tawalaya

@tormath1:matrix.org I dug a little deeper. It looks like the issue is due to a mismatch in expected topology and the provided topology in the /var/scaphandre. The VM I'm using gets 6 cores but the mounted topology only has 1 core. Resulting in the error message I'm seeing.

tormath1

@tormath1:matrix.org

[m]

nice investigation - that makes sense. In my case, the CPU used in the VM is the one from the host.

Same topology then.

Sebastian Werner

@tawalaya

I looks like the issue stems from the fact that I give the VM 6 of the 48 cores from the host but virsh pretends that each core is a socket in the vm thus creating the mismatch. I tried to use the cpu topology feature to basically get 1 cpu with 6 cores/threads but that dose not seem to work well. (https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/sect-virtualization_tuning_optimization_guide-virt_manager-cpu_options#sect-option_CPU_topology)

Sebastian Werner

@tawalaya

However, that seems to be normal for qemu -- https://superuser.com/questions/1577756/qemu-cpu-topology-doesn-t-match-maximum-vcpu-count I guess that's enough content for github issue ;)

tormath1

@tormath1:matrix.org

[m]

yep, nice findings :D

bpetit

@bpetit

Hi @tawalaya @tormath1:matrix.org thanks for raising the issue. It seems to be the same as hubblo-org/scaphandre#133 as you mentionned it in your PR

I answered to the PR so we could continue the discussion here : hubblo-org/scaphandre#207

Sebastian Werner

@tawalaya

@bpetit sorry for the delayed response - I agree with your assessment that it should be done in the qemu exporter, although I couldn't really see a path for that without resorting to somehow getting and reading the VM XML. But that might be because my lack of Rust skills :)

2 replies

bpetit

@bpetit

Hi @all !

The SDIA is organizing 3 events/hackathons/ideathons around evaluation of ICT environmental impacts, based on open-source tools (including scaphandre !).

Those events could be accessed either in person or remotely, for free :

Amsterdam - Sept 16
Berlin - Sept 23
London - Sept 28

I'll be there this friday at AMS and remotely in Berlin. If some of you could and want to join it would be a great occasion to meet !

Cheers

MarwaSlimene

@MarwaSlimene

Salut, J’ai commencé à utiliser scaphandre pendant mon stage de fin d’études, j’ai réussi avec cet outil à mesurer la consommation énergétique d’un processus mais je n’ai pas compris quelle est la différence entre la consommation calculer par scaphandre et la consommation afficher dans la barre des taches de mon bureau.
Pouvez-vous m’expliquer ce point afin de comprendre plus l’utilité de scaphandre ?

1 reply

bpetit

@bpetit

Hi @/all !

For those of you who are interested in having energy consumption evaluations in another context than physical machines running on x86, ie in the cloud, in VMs when you don't manage the hypervisor, etc... Some of you may have seen the hubblo-org/scaphandre#25 that talks about having a collaborative database with power consumption profiles per hardware. This project is taking the direction of a collaborative project managed by Boavizta : Energizta !

Documentation is WIP, but if you are interested in contributing, please tell me ! I'll send you the information to participate in the contributors meeting ! (spoiler alert, many amazing contributors onboard already, including researchers specialized in such topics !)

Arya [vern.cc]

@aryak:vern.cc

[m]

i tried the sample grafana json that is in the repo and all the measurements are in Mil

1 reply

how do i convert that to watts or some other common format?

or is milliwatt referred to as Mil

René Ribaud

@uggla

@bpetit please have a look at #234. Awx seems to have issues.

1 reply

Youssef Gharbi

@JoeGharbi_twitter

Hello everyone, @bpetit can you please check this issue #247 Thank you very much

14 replies

bpetit

@bpetit

Quick poll : on a scale from 1 to 10, how much would it be painful to you if I move this channel on a mattermost instance ? :)

3 replies

tormath1

@tormath1:matrix.org

[m]

what about a mattermost / matrix bridge?

On another project, we used a bridge matrix / IRC and it was working fine.

https://github.com/dalcde/matrix-appservice-mattermost

but it's certainly too much effort.

bpetit

@bpetit

I know it exists, but I'm still unclear if it would only require to install the bridge side by side with our mattermost instance, or if I also need a matrix instance ?

If it's just the bridge and it's quite simple to maintain, I'd be keen to do it

I see in the requirements :
"Matrix
A matrix server supporting the Application Services API is needed. No attempt has been made to track the minimum supported API version, but it should work with any reasonably modern server. It is assumed to be synapse in this document."

Is synapse a "public server" ? (warning, matrix-newbie around the corner ;) )

bpetit

@bpetit

okay it's the server itself ^^

tormath1

@tormath1:matrix.org

[m]

and on mattermost you can easily join a room or you need to be invited to the "org" (like on Slack)

bpetit

@bpetit

It seems we can create multiple teams on mattermost and invite to a team

olivier de Meringo

@demeringo

Hi @bpetit, could you please merge hubblo-org/scaphandre#254 when time permit ? This is just to make clippy happy with dev branch ;-)

Aurelien F

@aurelien619_twitter

Hi i'm trying to use scaphandre with proxmox. On hypervisor everything is OK. Now I'm tring to propagate metrics from hypervisor to vm , but with how to i don't succeded in it

I don't use libvirt so with https://hubblo-org.github.io/scaphandre-documentation/how-to_guides/propagate-metrics-hypervisor-to-vm_qemu-kvm.html#propagate-power-consumption-metrics-from-hypervisor-to-virtual-machines-qemukvm I can not give acces to vm for metrics

1 reply

Where communities thrive

People

Repo info

Activity