Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 15:45

    robertsjames on RJ-nT_detector

    S2 relative light yield re-done… (compare)

  • 15:29

    robertsjames on RJ-nT_detector

    Fix typo. (compare)

  • 15:19

    robertsjames on RJ-nT_detector

    Add cS1/cS2 to nT source; abili… (compare)

  • 14:28

    robertsjames on RJ-nT_detector

    Update width parameters at Matt… (compare)

  • 14:17

    robertsjames on RJ-nT_detector

    Make g1/g1_gas floatable. (compare)

  • 14:09

    robertsjames on RJ-nT_detector

    Deprecate extraction efficiency… (compare)

  • 13:31

    robertsjames on RJ-nT_detector

    Add coincidence level for nT, a… (compare)

  • 13:21

    robertsjames on RJ-nT_detector

    PEP8 fixes. (compare)

  • 13:07

    robertsjames on RJ-nT_detector

    Some fixes to handle multiple N… (compare)

  • 13:02

    robertsjames on RJ-ER_update

    PEP8 fixes. (compare)

  • 12:49

    robertsjames on RJ-ER_update

    Update NEST CI test. (compare)

  • 12:38

    robertsjames on RJ-ER_update

    It worked - do the same for NR> (compare)

  • 12:31

    robertsjames on RJ-ER_update

    See if removing ER nuisance par… (compare)

  • 12:13

    robertsjames on RJ-ER_update

    Fix typecasting issues. (compare)

  • 10:39

    robertsjames on RJ-ER_update

    Fix to coincidence level of 0. … (compare)

  • Aug 11 17:29

    robertsjames on RJ-ER_update

    Rename min_photons to coin_leve… (compare)

  • Aug 11 16:54

    robertsjames on RJ-ER_update

    Fix typecasting issue. (compare)

  • Aug 11 16:47

    robertsjames on RJ-ER_update

    Turns out method before was TF-… (compare)

  • Aug 11 14:46

    robertsjames on RJ-ER_update

    No approximations needed! This … (compare)

  • Aug 11 13:11

    robertsjames on RJ-ER_update

    Temporary: attempt to put in fu… (compare)

Jelle Aalbers
@JelleAalbers
And so it begins. Welcome to the flamedisx gitter chat
I hope this will be a useful channel for people from different institutions that are developing, using or just getting started with flamedisx
Thanks @sophiaandaloro for the suggestion
Sophia Andaloro
@sophiaandaloro
!! Yay! I'll post my several first questions soon :)
Sophia Andaloro
@sophiaandaloro

I've read the paper but haven't yet worked through the notebook tutorials. But I have some basic questions about the methods before I get started.

  1. Are there any other observables we're missing to add to each event? I assume not, but are x, y, z, t, and S1 S2 proven to be the best at reducing background contamination at 50% acceptance? I'm wondering if radial coordinates perform equally well or not, and I assume not but didn't see evidence (yet) for that.
  2. In all honesty, how was this method received in the community? It seems to be revolutionary to me, given the great benefit running on GPUs has in this particular algo, and getting the derivatives and Hessians is a nice addition. I guess what's stopping groups from picking this up and using it, if it boasts 1 year shorter discovery time?? Seems like we should all be hopping on this train.
  3. When you mean parameters tracking NEST values, do you want to initialize your model to look like NEST, or is there something I'm missing? I'm assuming you just mean taking NEST's functions for things like recombination, extraction etc and staying up-to-date with the latest data, but how else could we streamline these together? Asking as I'm one of our NEST correspondents :)
  4. There are many "nuisance" or latent parameters, right? So in some way we work around this, but we're just accounting for unknowns with them (or uncertainties? or both?) Can these somehow translate into learnable science, like improved detector response functions, or must they always be a nuisance?

As I run through the tutorials in coming days I think I might answer some of these myself, but let me know if you have any answers to them!

Jelle Aalbers
@JelleAalbers
Hi Sophia, thanks for your questions!
  1. Are there any other observables we're missing to add to each event? I assume not, but are x, y, z, t, and S1 S2 proven to be the best at reducing background contamination at 50% acceptance? I'm wondering if radial coordinates perform equally well or not, and I assume not but didn't see evidence (yet) for that.
In flamedisx all your functions can depend on as many observables as you like. If you have an (r, phi) map you can use it directly as a function of (r, phi) without converting it. (x, y, z, t) play no [particularly special role; we could have written bold x or something.
Jelle Aalbers
@JelleAalbers

In all honesty, how was this method received in the community? It seems to be revolutionary to me, given the great benefit running on GPUs has in this particular algo, and getting the derivatives and Hessians is a nice addition. I guess what's stopping groups from picking this up and using it, if it boasts 1 year shorter discovery time?? Seems like we should all be hopping on this train.

Glad to hear you like it :-). We spoke with people from XENON and LZ who both seemed interested. Both collaborations are focusing on getting a first result out asap, so I don't blame people at all for not jumping on a new tool right away. There are also still some issues that could block full use of flamedisx (e.g. https://github.com/FlamTeam/flamedisx/issues/68), though we hope to work on that soon

When you mean parameters tracking NEST values, do you want to initialize your model to look like NEST, or is there something I'm missing?

Yes, exactly. In an ideal world we would use the exact same basic parametrization and equations, but that is going to be difficult since we made a few changes to make the model easy to compute analytically (e.g. gauss -> beta for recombination fluctuation, no exciton/ion split, etc). We can't just call NEST functions from inside flamedisx either, even from nestpy, since we have to stay in tensorflow-land for the autoderivation to work. So having default yields/fluctuations to closely match NEST seems the best we could do. I think Jordan Palmer from Royal Holloway has started to look into this, let me add him to the chat...

Jelle Aalbers
@JelleAalbers

There are many "nuisance" or latent parameters, right? So in some way we work around this, but we're just accounting for unknowns with them (or uncertainties? or both?) Can these somehow translate into learnable science, like improved detector response functions, or must they always be a nuisance?

Indeed, LXe response models can have many nuisnance parameters. When you go above 5 or so params that actually change the band shape, fitting with standard methods becomes very difficult (as we discuss in the paper). I hope flamedisx will help here. With the better scaling with number of parameters we could try using more flexible functions such as high-degree polynomials or even neural nets for some functions like charge yields... We'd need a lot of great data to fit those well of course.

Sophia Andaloro
@sophiaandaloro
Thanks this is great. I'm glad I'm on the right track. There could be some way of "confirming" one's results against NEST's and warning a user if we stray outside of the bands of 2sigma safety, or something. As far as actual tracking though I see why we are limited. It is interesting to think, though, if we can match or replicate NEST with different, more convenient functions, what that implies about our assumptions on the true physical model. Logistically, something that bothers me in NEST is that we have 12 nuisance parameters, and Arianna of XENON actually found that they aren't used in the alpha modeling at all. So perhaps we find NEST is too parameterized too :-)
Thanks for all the explanations, I'll carry on with the notebooks and let you know if more come up. This chat room is helpful!
Jelle Aalbers
@JelleAalbers
No problem! Twelve nuisance parameters doesn't sound so bad considering the broad scope of NEST, and maybe some affect only certain processes as you mentioned. Flamedisx's ambitions are also more modest than NEST, we just offer a framework to fit detector data accurately, not predict the response before you build it. For fitting we (at least in XENON) allow departures from NEST, but it's a good point that large departures would be cause for concern as NEST keeps getting better.
We just released flamedisx v1.1.0, mostly for some updates to the optimizer stability and an experimental new limit-setting method; see https://github.com/FlamTeam/flamedisx/releases/tag/v1.1.0 for details
Chris Tunnell
@tunnell
Where does this fit in with other tools like https://pypi.org/project/pyhf/
Jelle Aalbers
@JelleAalbers
With 'this' do you mean flamedisx in general or our latest release? Pyhf looks like a great way of standardizing general binned likelihood inference, though of course it does not include LXe specific models like flamedisx does. Flamedisx is exclusively unbinned, while pyhf's examples seem to be binned only. Maybe it could also do unbinned though? At the very least you could approximately store an unbinned model as a binned model with very fine bins (in the limit of binsize -> 0 they are equivalent). For the 6D flamedisx model this would be a huge thing though.
Chris Tunnell
@tunnell
Ah, so histogram focused. I guess for most particle physicists who just run Geant4, that makes sense. Was just trying to figure out if there were similar packages out there.
Chris Tunnell
@tunnell
What about zfit https://pypi.org/project/zfit/? I'm just wondering if we can sustain this package long term, or if makes sense to try to merge with another
Chris Tunnell
@tunnell
There have just been a lot of talks in this realm at PyHEP, including auto-differentiation. Just trying to understand what makes most sense thinking of NEST internal fits and later for XENONnT. https://indico.cern.ch/event/882824/timetable/#20200713.detailed
Jelle Aalbers
@JelleAalbers

We consider zfit as a possible replacement for our inference code, see this issue: FlamTeam/flamedisx#41.

Indeed, many interesting talks at pyHEP. Most of flamedisx is a LXe response model, so that core will not be easily replaced, unless there is a generic 'frequentist hierarchical model integrator' somewhere where you can put in the model more easily. But the inference part is quite generic, not so different from blueice even; this could certainly be outsourced.

Chris Tunnell
@tunnell
Ah, so this came up when Bart was talking at the last PyHEP
Jelle Aalbers
@JelleAalbers
Yes, Bart and Jonas Eschle chatted bit, and Jonas opened this issue. At the time zfit didn't support tensorflow 2, but that got added some time ago (and more besides)
Jelle Aalbers
@JelleAalbers
Hi all, the flamedisx paper has just been published in PRD! https://journals.aps.org/prd/abstract/10.1103/PhysRevD.102.072010
Sophia Andaloro
@sophiaandaloro
Congrats! This is a great paper, happy to see it published.
Sophia Andaloro
@sophiaandaloro
Here I have come crawling back to ask more flamedisx questions ;) Specifically, can I create some sort of extra dependency in flamedisx i.e. in the s1 and s2 generation for instance? And would I do this by building my own source entirely with the new "blocks" I've made?
Also, I see by default we have 1t parameters in the blocks for the default ER and NR sources. In practice we would want to make our own, but I wonder if we could instead do something like a detector "file" kind of like done in NEST here that could help track all detector-specific parameters?
Jelle Aalbers
@JelleAalbers
Hi Sophia, sorry for the late reply! In FlamTeam/flamedisx#103, we'll introduce the defaults argument that allows you to change all defaults in bulk. So you could specify them in a config file (of any type you like), load them into a dictionary, then feed them to fd.Likelihood(..., defaults=big_config_dict)

For 'adding an extra dependency', it depends whether it's an observable or a new hidden variable.

For the first, you can let model functions take any new observable as a positional argument (just like you'd do for x, y, z, or t). You'll also want to make sure the new observable is produced during simulation and/or annotation . An example would be how https://github.com/FlamTeam/flamedisx/blob/master/flamedisx/xenon/x1t_sr0.py introduces the s2_relative_ly observable, which is a function of x_observed instead of x. (As it happens, we were too lazy to actually add the field distortion maps in random_truth, but I hope it's clear where you'd put them if you want to)

Jelle Aalbers
@JelleAalbers
Adding new hidden variables corresponds to changing the structure more fundamentally; for this you need to add or modify the blocks. For example, @jordan-palmer and Robert James are working to add a block to do an ion/exciton split before the photon/electron split. Some more guidance on how to work with blocks is in https://flamedisx.readthedocs.io/en/latest/blocks.html
Sophia Andaloro
@sophiaandaloro
Awesome, thanks for these explanations! Nice to know the configs are easy to change.
Jelle Aalbers
@JelleAalbers
I just sent out the invitation below for a first virtual flamedisx meeting. If you're following this channel, please consider yourself invited too! I'll post the when & where at the end of the week.

Hi all,

As flamedisx is maturing into a ready-for-use tool, let's have our first virtual meeting next week!

For this initial meetup, the aim would be to introduce ourselves, become aware of what everyone is doing and planning to do, and discuss how we'd like to communicate going forward.

In future meeting, we could discuss deeper issues, such as how we'll manage the git repository together, how we see our relation to NEST developing, etc.

If you'd like to attend, please fill out this WhenIsGood by Friday, 26 February: https://whenisgood.net/5ehy2dj

We live too far apart for normal office hours to intersect, so please be flexible if you can.

Feel free to forward this (by email or slack) to others in XENON/LZ that may be interested; they are all welcome to join.

Best,
Jelle

Jordan Palmer
@jordan-palmer
Hi all, does anyone have a good way of modifying the energy range of a source from a python script/notebook? For example, if I want a mono energetic source like Xe131m (164 keV), how can I change the energy spectrum from within my class instance without delving into the core code?
class Xe131m(fd.NRSource):
Sophia Andaloro
@sophiaandaloro
Hi Jordan, I think you can use fix_truth=dict(energy=5.) at the simulations stage, not sure if that works for class too. Or you can just override the default energies by re-specifying your energies like this (change for a constant to whatever you want: energies = tf.cast(tf.linspace(0.7, 150., 100), fd.float_type())
Cristian Antochi
@CristianAntochi
Hi @jordan-palmer, as Sophia said, if you define the class in a notebook, you can change the energy spectrum as energies = .... This works for me.
Jordan Palmer
@jordan-palmer
thanks a lot both of you. I was actually doing this but was misinterpreting the energy in the output data frame. Forgot there are set S1/S2 ranges in the core code
Jelle Aalbers
@JelleAalbers

Just released flamedisx v1.4.0. Thanks everyone for contributing fixes and updates, and for reporting bugs!

This release contains an important bugfix, FlamTeam/flamedisx#110. I would encourage everyone to update when they can. For a full lists fixes and updates, see https://github.com/FlamTeam/flamedisx/blob/master/HISTORY.md. Also note the documentation (https://flamedisx.readthedocs.io) has two new tutorials now.

Jelle Aalbers
@JelleAalbers

We're having a first flamedisx virtual meeting tomorrow (Friday 12 March) at 16:00 CET / 15:00 GMT / 10:00 ET / 09:00 CT / 07:00 PT.

Everyone interested in flamdeisx is welcome to join! Please PM me for the zoom link, as this is a public channel.

Agenda:

  • Introductions
  • How shall we communicate going forward?
  • Ongoing flamedisx work / plans (from those who would like to share something)