I've read the paper but haven't yet worked through the notebook tutorials. But I have some basic questions about the methods before I get started.
- Are there any other observables we're missing to add to each event? I assume not, but are x, y, z, t, and S1 S2 proven to be the best at reducing background contamination at 50% acceptance? I'm wondering if radial coordinates perform equally well or not, and I assume not but didn't see evidence (yet) for that.
- In all honesty, how was this method received in the community? It seems to be revolutionary to me, given the great benefit running on GPUs has in this particular algo, and getting the derivatives and Hessians is a nice addition. I guess what's stopping groups from picking this up and using it, if it boasts 1 year shorter discovery time?? Seems like we should all be hopping on this train.
- When you mean parameters tracking NEST values, do you want to initialize your model to look like NEST, or is there something I'm missing? I'm assuming you just mean taking NEST's functions for things like recombination, extraction etc and staying up-to-date with the latest data, but how else could we streamline these together? Asking as I'm one of our NEST correspondents :)
- There are many "nuisance" or latent parameters, right? So in some way we work around this, but we're just accounting for unknowns with them (or uncertainties? or both?) Can these somehow translate into learnable science, like improved detector response functions, or must they always be a nuisance?
As I run through the tutorials in coming days I think I might answer some of these myself, but let me know if you have any answers to them!
In flamedisx all your functions can depend on as many observables as you like. If you have an (r, phi) map you can use it directly as a function of (r, phi) without converting it. (x, y, z, t) play no [particularly special role; we could have written bold x or something.
- Are there any other observables we're missing to add to each event? I assume not, but are x, y, z, t, and S1 S2 proven to be the best at reducing background contamination at 50% acceptance? I'm wondering if radial coordinates perform equally well or not, and I assume not but didn't see evidence (yet) for that.
In all honesty, how was this method received in the community? It seems to be revolutionary to me, given the great benefit running on GPUs has in this particular algo, and getting the derivatives and Hessians is a nice addition. I guess what's stopping groups from picking this up and using it, if it boasts 1 year shorter discovery time?? Seems like we should all be hopping on this train.
Glad to hear you like it :-). We spoke with people from XENON and LZ who both seemed interested. Both collaborations are focusing on getting a first result out asap, so I don't blame people at all for not jumping on a new tool right away. There are also still some issues that could block full use of flamedisx (e.g. https://github.com/FlamTeam/flamedisx/issues/68), though we hope to work on that soon
When you mean parameters tracking NEST values, do you want to initialize your model to look like NEST, or is there something I'm missing?
Yes, exactly. In an ideal world we would use the exact same basic parametrization and equations, but that is going to be difficult since we made a few changes to make the model easy to compute analytically (e.g. gauss -> beta for recombination fluctuation, no exciton/ion split, etc). We can't just call NEST functions from inside flamedisx either, even from nestpy, since we have to stay in tensorflow-land for the autoderivation to work. So having default yields/fluctuations to closely match NEST seems the best we could do. I think Jordan Palmer from Royal Holloway has started to look into this, let me add him to the chat...
There are many "nuisance" or latent parameters, right? So in some way we work around this, but we're just accounting for unknowns with them (or uncertainties? or both?) Can these somehow translate into learnable science, like improved detector response functions, or must they always be a nuisance?
Indeed, LXe response models can have many nuisnance parameters. When you go above 5 or so params that actually change the band shape, fitting with standard methods becomes very difficult (as we discuss in the paper). I hope flamedisx will help here. With the better scaling with number of parameters we could try using more flexible functions such as high-degree polynomials or even neural nets for some functions like charge yields... We'd need a lot of great data to fit those well of course.
We consider zfit as a possible replacement for our inference code, see this issue: FlamTeam/flamedisx#41.
Indeed, many interesting talks at pyHEP. Most of flamedisx is a LXe response model, so that core will not be easily replaced, unless there is a generic 'frequentist hierarchical model integrator' somewhere where you can put in the model more easily. But the inference part is quite generic, not so different from blueice even; this could certainly be outsourced.
defaults argument that allows you to change all defaults in bulk. So you could specify them in a config file (of any type you like), load them into a dictionary, then feed them to fd.Likelihood(..., defaults=big_config_dict)
For 'adding an extra dependency', it depends whether it's an observable or a new hidden variable.
For the first, you can let model functions take any new observable as a positional argument (just like you'd do for x, y, z, or t). You'll also want to make sure the new observable is produced during simulation and/or annotation . An example would be how https://github.com/FlamTeam/flamedisx/blob/master/flamedisx/xenon/x1t_sr0.py introduces the s2_relative_ly observable, which is a function of x_observed instead of x. (As it happens, we were too lazy to actually add the field distortion maps in random_truth, but I hope it's clear where you'd put them if you want to)
Hi all,
As flamedisx is maturing into a ready-for-use tool, let's have our first virtual meeting next week!
For this initial meetup, the aim would be to introduce ourselves, become aware of what everyone is doing and planning to do, and discuss how we'd like to communicate going forward.
In future meeting, we could discuss deeper issues, such as how we'll manage the git repository together, how we see our relation to NEST developing, etc.
If you'd like to attend, please fill out this WhenIsGood by Friday, 26 February: https://whenisgood.net/5ehy2dj
We live too far apart for normal office hours to intersect, so please be flexible if you can.
Feel free to forward this (by email or slack) to others in XENON/LZ that may be interested; they are all welcome to join.
Best,
Jelle
class Xe131m(fd.NRSource):energies = tf.cast(tf.linspace(0.7, 150., 100), fd.float_type())
Just released flamedisx v1.4.0. Thanks everyone for contributing fixes and updates, and for reporting bugs!
This release contains an important bugfix, FlamTeam/flamedisx#110. I would encourage everyone to update when they can. For a full lists fixes and updates, see https://github.com/FlamTeam/flamedisx/blob/master/HISTORY.md. Also note the documentation (https://flamedisx.readthedocs.io) has two new tutorials now.
We're having a first flamedisx virtual meeting tomorrow (Friday 12 March) at 16:00 CET / 15:00 GMT / 10:00 ET / 09:00 CT / 07:00 PT.
Everyone interested in flamdeisx is welcome to join! Please PM me for the zoom link, as this is a public channel.
Agenda:
- Introductions
- How shall we communicate going forward?
- Ongoing flamedisx work / plans (from those who would like to share something)