Scikit-HEP/community

Scikit-HEP project - general community channel

Markus Löning

@mloning

It's not really a usage question, we're thinking about adopting awkward array as the data container for our package, but it's probably best if I come back with a more concrete list of questions

Jim Pivarski

@jpivarski

I just created this: https://gitter.im/Scikit-HEP/awkward-array I've been thinking about a specific room for Awkward Array for a while, and I think it makes the most sense to put it under SciKit-HEP like this.

Hans Dembinski

@HDembinski

@lnevay Henry already explained it well, yes, you can make histograms which are variable in the number and types of the axes. Please have a look at the docs, which should answer all questions you may have (if not, please drop a note so that I can improve). The Getting Started quick guide has an example here https://www.boost.org/doc/libs/develop/libs/histogram/doc/html/histogram/getting_started.html#histogram.getting_started.3d_histogram_axis_configuration_

Boost.Histogram relys on boost::variant2, so that’s always available; if you are in C++17, you can use std::variant if you prefer.

Henry is not correct here. You need to use the builtin boost::histogram::axis::variant. This is a special variant type with an axis interface. Other variants do not work.

Cédric Hernalsteens

@chernals

@HDembinski Thanks for sharing. We (my team working with @lnevay ) apparently missed that … In the end we found a workaround by creating a virtual base class implementing an interface similar to what we have already for the (lower) dimensional root histograms, and then a series of concrete classes (templated) for the various axes types.

I believe that, as we have a small number of types so far, this is manageable and there is less overhead.

Hans Dembinski

@HDembinski

@chernals That would work as well. When you fill the histogram, do you use the histogram::operator() or the fill method?

Cédric Hernalsteens

@chernals

histogram::operator()

Hans Dembinski

@HDembinski

If you can (because you have chunks of data to fill at once, not single values), the fill method is preferred, it is much faster. Especially when you use a virtual base class.

Cédric Hernalsteens

@chernals

@HDembinski Alright, thanks for the info! I’ll change that.

Hans Dembinski

@HDembinski

Much faster means: a factor 2 to 4 under ideal circumstances, and possibly more in your case because of the virtual function call

https://www.boost.org/doc/libs/develop/libs/histogram/doc/html/histogram/benchmarks.html

There are also still some optimizations that we can use to make .fill faster in the future, while histogram::call is already as fast as it can be

Cédric Hernalsteens

@chernals

Impressive !

Matthew Feickert

@matthewfeickert

Tonight I got this warning from pip

ERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

tensorflow 2.3.0 requires numpy<1.19.0,>=1.16.0, but you'll have numpy 1.19.1 which is incompatible.
tensorflow 2.3.0 requires scipy==1.4.1, but you'll have scipy 1.5.2 which is incompatible.

and from going to read about it on the PyPA's website this is going to be a major change come October. https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020

This has the potential to be a huge pain.

Hans Dembinski

@HDembinski

@matthewfeickert I am not sure yet whether this will affect me

Dear all, I am preparing another iminuit release with a new fit result display. This is a very visible part of iminuit, so feedback on the new display is appreciated. The update was triggered by request scikit-hep/iminuit#449 to add a warning when a fitted parameter is at the limit.

I prepared a short slide show that compares old vs. new, if you have comments, please let me know.

Henry Schreiner

@henryiii

be a huge pain

Being more strict is generally a little painful at first, but the old behavoir of installing incompatible versions was incorrect, so shouldn’t it be better?

At a quick glance, the new display looks great.

Are the red and green colors far enough apart to be distinguished with red/green colorblindness?

Hans Dembinski

@HDembinski

colorblindness

That is an issue I have never checked. The new displays also indicate an error by CAPITAL LETTERS, so it is at least better than before

Is there an easy way (a filter perhaps) to apply to see how it would look for a colorblind person?

Henry Schreiner

@henryiii

Ahh, good. The capital letters may be enough. Before you could scan true/false, so wanted to make sure it’s still easy to pick out failures if you can’t see colors well (the new system is cleaner and explains what it wrong much more clearly)

Hans Dembinski

@HDembinski

Ok, Gimp can do it although the option is fairly hidden. The display also works for colorblindness

Henry Schreiner

@henryiii

I (just) downloaded an app that lets you do that (color blind pal). If I trust the three deficiancy filters, that’s the worse one.

Hans Dembinski

@HDembinski

It looked a bit better in GIMP

Henry Schreiner

@henryiii

There are three types of colorblindness, the other two showed up better.

Hans Dembinski

@HDembinski

I know and I mean the same kind you are looking at

Henry Schreiner

@henryiii

Hans Dembinski

@HDembinski

That's gimp

Henry Schreiner

@henryiii

They are just approximations. Ideally we could get someone to actually look at it. Maybe you could tweak the brightness just a tiny bit on either the green or the red?

Hans Dembinski

@HDembinski

Sure

Henry Schreiner

@henryiii

The colors are slighly more important in the new display, which is why it’s a good idea to optimize a little now.

Hans Dembinski

@HDembinski

The colors are less important in the new display

The new display tells you in words what's wrong and it uses CAPITAL LETTERS when there is a problem

Henry Schreiner

@henryiii

I mean to quickly detect that something failed. They are less important if you already are looking at it and reading the text (which is great).

Hans Dembinski

@HDembinski

It is not possible to get more contrast easily without changing the color of the red to a violet hue, which looks weird if you are not colorblind

Henry Schreiner

@henryiii

What about making the green a bit lighter?

Hans Dembinski

@HDembinski

Then it will look unpleasant

I mean to quickly detect that something failed.

Me, too. The CAPITAL LETTERS are easy to spot

Henry Schreiner

@henryiii

Don’t make it too “bright”, keep some light grey mixed in. EDM is capital even if everything succeeds. What about a little “FAILED” label at the bottom just below the table if any items failed?

hey Henry - yep, I have no problem with this display and I am red/green colorblind

Awesome, let’s go with it as is, then!

Matthew Feickert

@matthewfeickert

be a huge pain

Being more strict is generally a little painful at first, but the old behavoir of installing incompatible versions was incorrect, so shouldn’t it be better?

Sorry I was being very brief last night. In general I think that this change is great! I was more selfishly thinking of the fact that TensorFlow is going to force a lower version of SciPy then the one we want to use for pyhf as the more recent releases benefit from some nice FORTRAN bug fixes

Hans Dembinski

@HDembinski

This choice supports various kinds of colorblindness. I like the old colors better, but improving the situation for colorblindness is a valid concern

alexander-held

@alexander-held

when running this in a terminal without all the colors, another thing that could help is something that breaks the usual format of the table (e.g. some indented warning in a new line saying not everything is 100% fine)

(this could also be implemented by the user themselves, I personally find it much easier to look for some loud text that only appears in case of issues than to scan the text in the table when fitting many times in a row)

Matthew Feickert

@matthewfeickert

Updated my pip Gist to explain why this change is good: https://gist.github.com/matthewfeickert/78cd03376435c05b6104f376cd43b537

Where communities thrive

People

Repo info

Activity