Scikit-HEP/community

Scikit-HEP project - general community channel

Henry Schreiner

@henryiii

Can I get a ~~guinea pig~~ volenteer to try https://henryiii.github.io/histogram-tutorial/ before the PyHEP meeting in 30 mins?

Vidya Sagar V

@vidya:vidyasagarv.com

[m]

image.png

It works!

Henry Schreiner

@henryiii

And the histogram talk is in 30 mins: https://indico.cern.ch/event/1133099/

Sweet, thanks!

(Binder is literally still building since before I started posting until now)

2 replies

Aman Goel

@amangoel185:matrix.org

[m]

Yeah, it is painfully slow

Hans Dembinski

@HDembinski

I am also cross-posting this here.

Since I had a discussion about iminuit performance recently, so I ran some benchmarks to compare iminuit + numba_stats (both my libraries) with RooFit.
numba_stats has faster implementations of statistical distributions than scipy. I am very pleased to find that iminuit outperforms RooFit by a large margin on a fit of a Gaussian. I did not expect the gain to be so dramatic. RooFit in parallel mode is very slow compared to numba in parallel mode. You can see this in the bottom right of the following plot. The x-axis is the number of points in the fitted data sample, the y-axis is the runtime of the fit.

5 replies

This message was deleted

This once gain demonstrates how powerful numba is and that writing in C++ does not guarantee you good performance
You read more about this benchmark in the iminuit docs https://iminuit.readthedocs.io/en/latest/benchmark.html

Side remark: I got to this point not only by simply adding numba.njit on every function. I added a small series of patches in the last weeks to numba_stats and iminuit to improve the performance to this point, and I used my acquired knowledge about writing fast numerical code in C++ and Python.

Hans Dembinski

@HDembinski

@eduardo-rodrigues Perhaps we can get more people interested in trying out iminuit + scikit-hep with plots like this?

Matthew Feickert

@matthewfeickert

Interesting to see there's a HSF/IRIS-HEP "Matplotlib training" coming up that will have a focus on mplhep. :+1: https://indico.cern.ch/event/1058838/

Eduardo Rodrigues

@eduardo-rodrigues

@eduardo-rodrigues Perhaps we can get more people interested in trying out iminuit + scikit-hep with plots like this?

I think it can only help, yes. The examples code are excellent.

Angus Hollands

@agoose77:matrix.org

[m]

@matthewfeickert: isn't mamba the default now for repo2docker?

Henry Schreiner

@henryiii:matrix.org

[m]

I thought it was

Chris Burr

@chrisburr

I also thought it was, though if it's still slow it's possible they're hitting IO limits

Matthew Feickert

@matthewfeickert

So this is still pretty experimental, and won't be out in what we really consider "public facing" until we release pyhf v0.7.0, but the pyhf docs now have Pyodide/JupyterLite in them. :) Hopefully this will be the start of interactive docs, where we have static examples with live runnable versions side-by-side.

Eduardo Rodrigues

@eduardo-rodrigues

Wow, cute :+1: !

Henry Schreiner

@henryiii

Interesting, since there’s a “plt.show()”, I get a warning "UserWarning: Matplotlib is currently using module://matplotlib_inline.backend_inline, which is a non-GUI backend, so cannot show the figure.” when pasting the example. But it shows up anyway (via inline).

3 replies

Chris Burr

@chrisburr

Nice! For the second example the requests.get("https://git.io/JJYDE").json() line fails with an issue importing SSL, is there any workaround for that? Or is network access just not an option from within wasm?

8 replies

Henry Schreiner

@henryiii

Network access is there, but it’s async only, so the stdlib doesn’t work, and therfore requests doesn’t work.

import pyodide, json
inp = pyodide.open_url("https://raw.githubusercontent.com/scikit-hep/pyhf/master/docs/examples/json/2-bin_1-channel.json”)
wspace = pyhf.Workspace(json.load(inp))

Works.

Matthew Feickert

@matthewfeickert

Though that will change next month(hopefully) https://gitter.im/Scikit-HEP/community?at=623dda81161ffc40d7d133d1

Iason Krommydas

@iasonkrom

@HDembinski
May I suggest a small iminuit feature that I would find useful?
I think in methods like mnprofile and mncontour It would be really nice to have the ability to scan on points that are evenly spaced on a log scale.
I had to do this recently using iminuit because one fitting parameter was the thermal cross section for a generic WIMP. Of course I wrote a for loop myself to do this but it would be nice to have it built in since you already have the linear scanning.

2 replies

Or maybe just allow the user to choose the points as input so they can use np.logspace for that.

if that already exists and I'm completely blind and don't see it in the documentation then I'm sorry.

bretttully

@bretttully

Hi all, I have been using conda for a long time to build pybind11 c++ extensions. There is a push in my team to move away from conda so I am exploring the best approach to build the c++ extensions with other tools. Specifically, if I have dependencies on things like boost and opencv in my C++, what is the best way to define those dependencies in a non-conda build? Typically, I would just get those c++ dependencies from conda-forge and move on. I notice your docs on packaging do not talk about conda so I assume you have some experience with this. Any suggestions on places to read? Thanks!

6 replies

Matthew Feickert

@matthewfeickert

@henryiii @HDembinski @jpivarski I haven't given this much thought if this would actually be useful or not, but if we could get a package index up somewhere (similar to https://anaconda.org/scipy-wheels-nightly, though not necessarily on Anaconda Cloud as I don't think they'd give us free space) do you see any benefit in doing nightly builds and uploads of wheels? It would make testing against HEAD for boost-histogram, awkward, and iminuit easier in general for people as they don't need to build anything and if a similar approach as matplotlib/matplotlib#22733 was used then there wouldn't even need to be additional builds.

Thoughts?

17 replies

Eduardo Rodrigues

@eduardo-rodrigues

[@matthewfeickert - quick side comment to say that we're got a pyhf related question on the Scikit-HEP forum mailing list. I know the list is not used so often, hence this "ping".]

4 replies

Henry Schreiner

@henryiii

Pyodide 0.20 is out, now based on CPython 3.10, with boost-histogram! You can now use Hist directly in your webbrowser without any setup! It also includes support for exceptions from C++ (pybind11) in support of boost-histogram, and much more. It uses wheels direclty and pypa/build now, too. Jupyter-lite has not updated yet, but should be soon (they had a branch running the release canidate). In the near future I want to work on adding more Scikit-HEP libraries (started on iMinuit, also Awkward eventually, maybe for v2).

Angus Hollands

@agoose77:matrix.org

[m]

Thanks @henryiii ! What are the thoughts on the Awkward front - a separate awkward-wasm-kernels package? I can't remember where we left off with this conversation last time, but maybe if we get the kernel multiplicity down we could look moving to Rust for the existing CPU kernels too.

Henry Schreiner

@henryiii

It should be just the normal awkward package, compiled for wasm; it’s an archetecture, like x86, etc. 0.20 is the first version to support Rust, too, by the way. :)

Eventually PyPI might support this as a new tag, but that will take a while, and it’s not stable yet anyway (part of the reason it’s all compiled together, emscripten doesn’t make stability promises yet).

Angus Hollands

@agoose77:matrix.org

[m]

0.20 is the first version to support Rust, too, by the way.

Oh, nice!

Oh, I was thinking of the pure-python packaging support on e.g. PyPI/conda-forge, but I realise that we're not there yet w.r.t wheel tags etc.

Well, to be precise, I was thinking of a workaround to missing wheel tags by defining a separate package, but it's a moot point anyway if we don't have the rest of the necessary features in place 🙂

Henry Schreiner

@henryiii

Importing histoprint correpts the pyodide default terminal, working on fixes there. :/ Pretty sure the jupyter one will be fine.

Chris Burr

@chrisburr

There is also an effort to support wasm as an opt-in "platform" for conda-forge like how linux-aarch64/linux-ppc64le/osx-arm64 are handled (though handing fortran is tricky so it will take a while)

Angus Hollands

@agoose77:matrix.org

[m]

@henryiii: I really enjoy how many Python3.9+ features you've managed to leverage in https://github.com/henryiii/scikit-hep-repo-review

Henry Schreiner

@henryiii

And 3.10, too. ;)

Angus Hollands

@agoose77:matrix.org

[m]

That's certainly ... the case 🤣

Matthew Feickert

@matthewfeickert

I took the liberty of activating Scikit-HEP org level discussions https://github.com/orgs/scikit-hep/discussions.

Can be useful for broad topical discussions (If we really don't like this, my apologies, and we can turn it off)

5 replies

Jim Pivarski

@jpivarski

I was playing with a "unpin/pin" button in GitHub, wondering "What does this do?" and unpinned/repinned the scikit-hep/awkward-1.0 repo. That changed the order; I don't know if you have a preferred order, but if I goofed things up, go ahead and change the order back to the way you prefer. (Sorry! I thought it was going to pin a PR, and was wondering if that might be related to a feature that would let me prioritize issues within a repo.)

2 replies

Henry Schreiner

@henryiii

@agoose77:matrix.org How do you like it now? :D https://henryiii.github.io/scikit-hep-repo-review/

3 replies

Henry Schreiner

@henryiii

The Scikit-HEP developer pages now has a beta page to compare a GitHub repository against the developer guidelines! Runs in pyodide 0.20's Python 3.10 on your machine, it's just a default Jekyll Github Pages static site!
https://twitter.com/HenrySchreiner3/status/1516062487740878848?s=20&t=xTFNYLu4Hi0hh-80kGIF1A

Eduardo Rodrigues

@eduardo-rodrigues

:+1:

Jim Pivarski

@jpivarski

SciPy 2022 Birds-of-a-Feather Session proposals are open until June 15. Do we want to have one for Scikit-HEP/PyHEP/general Python in HEP?

https://www.scipy2022.scipy.org/bof-sessions

2 replies

Matthew Feickert

@matthewfeickert

https://github.blog/changelog/2022-04-25-git-io-deprecation/

This affects the pyhf docs, but I haven't checked yet if it affects any other Scikit-HEP tools. Probably worth a quick git grep on all your projects.

Henry Schreiner

@henryiii

$ all-repos-grep "git.io"
output/scikit-hep/pyhf:README.rst:   >>> wspace = pyhf.Workspace(requests.get("https://git.io/JJYDE").json())
output/scikit-hep/pyhf:src/pyhf/cli/infer.py:        $ curl -sL https://git.io/JJYDE | pyhf fit --value
output/scikit-hep/pyhf:src/pyhf/cli/infer.py:        $ curl -sL https://git.io/JJYDE | pyhf cls

Looks like PyHF is the only one. I generally don’t use URL shorteners. Though I can guess why you might be using it there. There’s also a all-repos extension that will replace git.io links automatically (probalby has to be done before it gets turned off, though, to get the replacement link)

2 replies

Matthew Feickert

@matthewfeickert

Nice to see you sharing this at PyCon Henry! https://twitter.com/willingcarol/status/1520415422784229376?s=21&t=agWgR7_iFkA-HVEjWIco2w

Where communities thrive

People

Repo info

Activity