HSF/PyHEP-histogramming

Discussions around histogramming

Hans Dembinski

@HDembinski

Had to figure out how to watch tags but I am watching it now

Thank you for adding the tag, @jpivarski !

N!no

@LovelyBuggies

@HDembinski Hans, could you please list some common-used HEP analysis methods?
I am thinking about adding some analysis shortcuts for Hist as mentioned in scikit-hep/hist#4.

I consider Scipy's optimize module a good solution, but not sure whether it's proper.

Jim Pivarski

@jpivarski

@henryiii @HDembinski Just as a heads-up, I'm not 100% sure that StackOverflow's "watch tags" feature works: sometimes when I visit the site, I find unanswered questions there that I don't remember choosing to skip. It doesn't seem to be emailing every question with a given tag—maybe it doesn't email if the tag is attached after the first edit? I don't know.

mu22le

@mu22le

@LovelyBuggies minuit is an "industry standard" in HEP. People know it, trust it, but also know its shortcomings. I have seen heated discussions about switching to a different minimiser. I think you should at least try to provide one or two minuit example.

N!no

@LovelyBuggies

@LovelyBuggies minuit is an "industry standard" in HEP. People know it, trust it, but also know its shortcomings. I have seen heated discussions about switching to a different minimiser. I think you should at least try to provide one or two minuit example.

Thanks for your suggestion. Seems good!

Hans Dembinski

@HDembinski

Full agreement with @mu22le

Also, iminuit is not really about the minimizer. In HEP, we want errors for our parameters and there is no other package which computes errors like MINUIT

Hans Dembinski

@HDembinski

@HDembinski Hans, could you please list some common-used HEP analysis methods?

I am not sure what you need.

For iminuit, you can steal from the basic tutorial

https://nbviewer.jupyter.org/github/scikit-hep/iminuit/blob/master/tutorial/basic_tutorial.ipynb

N!no

@LovelyBuggies

Thanks! It's mainly for Hist external package integration design. It seems that a lot of useful tools have been provided, pyhf and minuit. I'm going to go through their docs and try some integrations in the next few days.

Eduardo Rodrigues

@eduardo-rodrigues

@LovelyBuggies, could you provide here for the benefit of all what @henryiii and you are aiming for with the Hist package? I'm unsure from the issues you have been creating on the repo recently. Thanks.

Hans Dembinski

@HDembinski

Different topic: @jpivarski @lukasheinrich boost-histogram supports integer and category axes, which can and should be used to bundle related histograms together. I use these axes to have a common histogram that contains separately signal, background, different data subsets, etc. What can histbook do that boost-histogram with these axes cannot do?

mu22le

@mu22le

@LovelyBuggies if you want to see what HEP people do when fitting histograms you could take a look at these tutorials https://pastebin.com/LxF5vxxJ

N!no

@LovelyBuggies

@eduardo-rodrigues It's still under discussion, and I am waiting for henry to make the decision. Currently, I'm trying to get familiar with other HEP packages.

Eduardo Rodrigues

@eduardo-rodrigues

@LovelyBuggies OK, thanks. Makes sense.

N!no

@LovelyBuggies

When I looked back to build the boost-histogram dev env recently, I met a problem. Not sure why this happened. I removed the egginfo files and use pip -v -e .[all], but the Traceback was thrown.

Hint, I just tested the WIP branch henryiii-iweight and it was broken similarly. Does it have something to do with this trial? And should branch henryiii-iweightwork well?

Henry Schreiner

@henryiii

It’s complaining that you aren’t compiling with a C++14 compiler. Do you know what compiler you are using? It is on conda or not on conda?

I’ve been building software for a ventilator flow meter, so have been completely buried (and probably will be till the official demo at the Hospital on Monday)

Conda has it’s own compiler which works. A modern distribution should be fine as well. Older WIndows/Linux compilers may not.

N!no

@LovelyBuggies

It's not in conda environment, it's by pip install

It's odd because I have never encountered this before.

Henry Schreiner

@henryiii

What macOS version? macOS has supported C++14 for ages...

So regardless of wheter it’s in conda or not, it shouldn’t hit this...

Chris Burr

@chrisburr

I think the error is that it isn't finding a compiler at all

xcrun: error invalid active developer path

N!no

@LovelyBuggies

You are totally right!

I just realized that it's due to the naughty Catalina.

My clang was stolen for some reason, so my Clang is not available when I install the dev env.

Anyway, I have solved this problem and thank you!

Henry Schreiner

@henryiii

Haha, welcome to Catalina, land of many bugs! I like the improvements, but I think it holds the records for number of bugs. :)

N!no

@LovelyBuggies

😊

N!no

@LovelyBuggies

Btw, I wonder whether boost-histogram can pass the unit tests. Unluckily, I failed. :( Mainly two reasons concerning bh c++: a) the position arguments; b) no 'view()' for bhc.

(@henryiii, I see you are busy recently, so TAL at your convenience, it's not emergent.)

Henry Schreiner

@henryiii

Looks like you need to recompile

Maybe you compiled on the old branch then ran the tests on develop?

After upgrading macOS versions you need to update the command-line tools or XCode (if XCode, then you need to start it to trigger the command-line tools update).

N!no

@LovelyBuggies

Looks like you got the point, so I update my branch and retest. However, there is another problem -- INTERNALERROR.

N!no

@LovelyBuggies

And I think my command line tools are already up to date:

$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

Henry Schreiner

@henryiii

Wow, that’s probably not our fault. Try uninstalling pytest-sugar, it’s just an optional pretty printer for pytest

N!no

@LovelyBuggies

Yes, this is due to the pytest-sugar. In fact, the pytest-sugar is auto-installed when pip install -v -e .[all&dev], meaning that users and developers should manually uninstall it after activating the env and then reinstall dev env. So I think we should either make pytest-sugar compatible or just remove the pytest-sugar requirement in the editable mode.

N!no

@LovelyBuggies

Hint: pytest-sugar exists in 3 places in boost-histogram: a) METADATA; b) setup.py; c) CONTRIBUTING.md

Henry Schreiner

@henryiii

I think it should maybe be mentioned, but probably removed as a dev requirement.

donalrinho

@donalrinho

Hi all, I'm wondering if there is a way to quickly replace zeros in a bh.Histogram() with a finite value? I'm dividing two of these to make an efficiency, and end up with RuntimeWarning: invalid value encountered in true_divide coming from bins with no entries.

N!no

@LovelyBuggies

Maybe you can use an iteration like this:

hist = bh.Histogram(bh.axis.Regular(50, -3, 3)).fill(np.random.normal(size=10))
for idx in range(50):
     if hist[idx] == 0: hist[idx] == finite

You can also use numpy method to re-fill, but I don't think it's simpler than this :)

Henry Schreiner

@henryiii

Sorry, I intended to respond, but got distracted. If you treat them as numpy arrays, then you can do anything you would with numpy arrays:

with np.seterr(divide=‘ignore’):
   h1.view() / h2.view()

Or, even better,

h2_masked = np.ma.masked_array(h2.view(), 0)
h1.view() / h2_masked

That one will be properly masked out where the division is invalid.

Henry Schreiner

@henryiii

Or, to numpify N!no’s example, you could do:

hist[:] = np.where(hist.view() == 0, finite, hist.view())

which is the closest to what you asked but not always the best way to handle divisions by zero.

Where communities thrive

People

Repo info

Activity