Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 17 22:04
    ericpre commented #2985
  • Aug 17 21:57
    ericpre commented #2987
  • Aug 17 21:53
    omnivus commented #2985
  • Aug 17 21:53
    omnivus closed #2985
  • Aug 17 18:40
    michaelwuww reopened #2987
  • Aug 17 18:40
    michaelwuww closed #2987
  • Aug 17 18:40
    michaelwuww commented #2987
  • Aug 17 16:22
    jlaehne labeled #2977
  • Aug 17 11:12
    ericpre commented #2985
  • Aug 17 10:48
    ericpre labeled #2987
  • Aug 17 10:48
    ericpre commented #2987
  • Aug 17 09:47
    michaelwuww commented #2987
  • Aug 17 09:46
    michaelwuww commented #2987
  • Aug 17 08:43
    ericpre commented #2987
  • Aug 17 08:24
    ericpre edited #2987
  • Aug 17 07:41
    michaelwuww opened #2987
  • Aug 16 22:52
    codecov[bot] commented #2876
  • Aug 16 22:37
    codecov[bot] commented #2876
  • Aug 16 22:36
    CSSFrancis synchronize #2876
  • Aug 16 19:23
    codecov[bot] commented #2876
Wu Michael
@belleh1234567890_gitlab
the missing part contains the signals, which I had to remove so the fitting is not affected by them.
Wu Michael
@belleh1234567890_gitlab
image.png
Thomas Aarholt
@thomasaarholt
Can you send me a single spectrum? zip the file and put it here?
Wu Michael
@belleh1234567890_gitlab
Wu Michael
@belleh1234567890_gitlab
Here is the raw spectrum or were you asking for the subtracted one?
Wu Michael
@belleh1234567890_gitlab
in addition is there a way to increase the maxfev for curve fitting?
Thomas Aarholt
@thomasaarholt
m.fit(maxfev=10000)? :)
image.png
I'm guessing it's the bump at 0.06eV that you are interested in?
I've plotted the above on a log-scale
s.plot()
plt.xlim(-0.1, 0.1)
plt.ylim(50, None)
plt.yscale('log')
Thomas Aarholt
@thomasaarholt
import hyperspy.api as hs
s = hs.load("test_spectrum.hspy")
s = s.inav[0,0] # I select the only spectrum in the spectrum image, easier to work with


s = s.isig[0.02:0.1] # crop the signal so we don't have long noisy tails or negatives

m = s.create_model()
print(f"Using the default background, see m.components:\n{m.components}")
m.set_signal_range(0.020, 0.04)
m.fit()
#m.multifit() # if you are on a spectrum image and happy with your parameters and bounds

# Take a look at m.plot() here before the signal range is reset
# to understand what range is used for fitting
m.reset_signal_range()


# This might be enough for you:
s_background_subtracted = s - m.as_signal()
s_background_subtracted.data[s_background_subtracted < 0] = 0.


# Or we can continue by modelling the 0.06eV bump as a Gaussian
# Fix the background, since it only depends on the pre-edge before the 0.06 bump
# Create a gaussian component to model the 0.06eV bump
G = hs.model.components1D.Gaussian(centre=0.06, sigma=0.004)
G.centre.bmin = 0.05
G.centre.bmax = 0.07

background = m[0]
background.set_parameters_not_free()

m.append(G)

m.fit()
# m.multifit(bounded=True) # if you are on a spectrum image

m.plot(True)

# You can map out where the intensity is if you have done a spectrum image.
# Is not interesting on a single spectrum, but
# might look super cool if you have a map over a region :)
A_map = G.A.as_signal()

s_modelled_bump = m.as_signal(component_list=[G]) # export everything except background 
s_modelled_bump.plot()
image.png
Thomas Aarholt
@thomasaarholt
How does that look? The last plot is just a gaussian that hyperspy modelled on your 0.06eV bump.
If you are fitting this on a spectrum image where the bump goes up and down in intensity, then I would also add G.A.bmin.value = 0 as a lower bound on the Gaussian intensity.
Wu Michael
@belleh1234567890_gitlab
Wow it looks pretty damn good. I haven't had the time to try it yet. But this is extremely helpful!
Mingquan Xu
@Mingquan_Xu_twitter
Hi, I have a question about the "model fitting" in HyperSpy.
If I want to fit my curve with some fixed parameters, how to do it?
Such as I want to fit my curve with a certain Gaussian center value?
When I try to set up a certain value for the amplitude of Gaussian, HyperSpy told me like this:
The 'A' trait of a Gaussian instance must be a Parameter or None, but a value of 24557.137086863306 <class 'float'> was specified.
Wu Michael
@belleh1234567890_gitlab
Try setting A to be a not free parameter
BTW, how can I get the data for each model component?
I have used three Gaussian components to fit my data, and want to export them and plot them seperately.
Thomas Aarholt
@thomasaarholt

@Mingquan_Xu_twitter the issue there is that you are passing a value as the parameter instead of its value:

G = hs.model.components1d.Gaussian()
G.A = 24557.0 # error
G.A.value = 24557 # works!

The reason this happens is that the Parameter A is a class with a lot of properties, not only its value. It's also got a .std (standard deviation), .bmin and .bmax(lower and upper bounds), .map (its values in all the navigation space pixels) and more. If you set the value like you did, you would delete all of these properties that G.A has.

If you want to export each signal separately, you would use m.as_signal(component_list=[G]), where G is one of the components you want to export. Do this for each of your gaussians.
Mingquan Xu
@Mingquan_Xu_twitter

If you want to export each signal separately, you would use m.as_signal(component_list=[G]), where G is one of the components you want to export. Do this for each of your gaussians.

It works. Thank you very much!

Arthur Woll
@arwoll

Hi All: I'm excited to find this community of HyperSpy users having just discovered it as a possible, very nice solution for x-ray diffraction- and SAXS-microscopy.

Out of curiosity, is there any thought /motivation among developers to either combine or build extensions to translate between HyperSpec and xarray?

Thomas Aarholt
@thomasaarholt
Hi Arthur! Was "HyperSpec" just a typo, or is there a package you are referring to?
Arthur Woll
@arwoll
@thomasaarholt Ha! A typo. My apologies.
Thomas Aarholt
@thomasaarholt
The xarray package is really good, but I don't think there is any strong motivation right now to replace any of the numpy/axes-manager functionality with xarray. Are there any particular benefits to xarray that you were thinking of?
Arthur Woll
@arwoll
Thanks Thomas! No -- not at all. I am newly gaining familiarity with both and noticed the overlap in functionality -- especially related to axis handling (as your reply suggests).
OliDG
@OliDG

Hi!

I encounter a big problem with the quantification using EDS model, the background fitting has no effect on the quantification results... !
Here is the piece of code I use, I try to modify the parameters of the fit_background() to get different background modelling, I can see in the m_tco components the polynomial values of the background changing, but it has absolutely no effect on the quantification outcome...! Am I missing something obvious? I can't find any option to manually force a background correction, neither in the get_lines_intensity() nor in the quantification() instances.

#fit the background and plot the result
m_tco.fit_background(kind='single',start_energy=0.0, end_energy=10.0, windows_sigma=(2.0,1.0))
m_tco.plot(True)

#Get lines intensity from the model and quantify: background option?
m_int_fit_tco = m_tco.get_lines_intensity(xray_lines_selected)
composition_fit_tco = si_TCO.quantification(m_int_fit_tco,method='CL',factors=kfactors,plot_result=True)
these 2 gives fitting the same outcome...
Cu (Cu_Ka): Composition = 0.20 percent
Ga (Ga_Ka): Composition = 0.59 percent
In (In_La): Composition = 55.97 percent
Na (Na_Ka): Composition = 0.49 percent
O (O_Ka): Composition = 42.02 percent
Se (Se_Ka): Composition = 0.07 percent
Si (Si_Ka): Composition = -0.72 percent
Sn (Sn_La): Composition = 1.69 percent
image.png
Thomas Aarholt
@thomasaarholt
Could you provide a single spectrum and code?
1 reply
OliDG
@OliDG
Here you go: I only skipped the k-factor part to allow the code working:
#For Thomas:

##Sum spectrum saved using: 
    #si_TCO.sum().save('si_TCO_sum.hspy')
    ##Available here: https://github.com/OliDG/DataSample/blob/cd7d2a33cefe6389ccfe932897e4d4f0d2878c09/si_TCO_sum.hspy


#list of element and kfactors
#ITO
#elements = ['Ag', 'Cu', 'F', 'Ga', 'In', 'Mo', 'Na', 'O', 'Se', 'Si', 'Sn', 'Ti']
element2quant = ['Ag', 'Cu', 'Ga', 'In', 'Na', 'O', 'Se', 'Si', 'Sn']

si_TCO.set_elements(element2quant)
si_TCO.set_lines([])
xray_lines_selected=si_TCO.metadata.Sample.xray_lines

#auto k-factor picking in the original code, the kfactors parameter is hardcoded below for the example to work:
# #check the elements to quantify
# kfactors = [0] * len(si_TCO.metadata.Sample.xray_lines)

# #Automaticallyt pick up the right k-factors from the .csv file
# for i in range(len(si_TCO.metadata.Sample.xray_lines)):
#     kfactors [i] = kfactors_dic[si_TCO.metadata.Sample.xray_lines[i]]
    ##Here is the list I used for this one:

kfactors = [2.947362,
 1.834458,
 2.193666,
 3.090453,
 0.874487,
 1.179439,
 3.058373,
 1.0,
 3.474381]

#create a model and quantify
m_tco = si_TCO.sum().create_model()
print('model created')
#Fit the Xray lines
m_tco.fit(bounded=False,)
print('peak fit')
m_tco.plot(True)

#fit the background
#m_tco.free_background()
m_tco.fit_background(kind='single',start_energy=0.0, end_energy=20.0, windows_sigma=(5.0,4.0))
print('background fit')
m_tco.plot(True)

#Quantify
m_int_fit_tco = m_tco.get_lines_intensity(xray_lines_selected)
composition_fit_tco = si_TCO.quantification(m_int_fit_tco,method='CL',factors=kfactors,plot_result=True)
michaelwuww
@michaelwuww
Is there a way to change the fitting ranges during multifit? If not would creating a mask and multifit these regions separately the best way to go about this?
Arthur Woll
@arwoll

Hi All: I've had great luck playing with model fitting over the last week. I then found a problem that seemed ready-made for samfire. Following instructions on the web-site and from a previous thread, I created a samfire model. But when I issue the start() method, samfire appears to work for a while, generates a figure and a progress bar, but then quits with the progress bar remaining at 0.

samf=m2.create_samfire(workers=None, ipyparallel=False)
samf.metadata.goodness_test.tolerance = 0.3
samf.refresh_database()
samf.start(optimizer='lm', loss_function='ls', bounded=True, kind='smart')

I have tried this on two platforms (Mac - hyperspy version 1.7.1 and Linux (hyperspy version 1.7.0) with the same behavior. No errors are reported. Execution merely stops and samf.pixels_done returns 0. The data shape has dims (340, 20 | 90) and the model has 9 components, but with most parameters fixed or linearized. Any suggestions?

Thomas Aarholt
@thomasaarholt
@OliDG just checking I understood your problem: You observe that the component intensities don't change after you refit the background component, correct?
The reason this is happening is that the values (e.g. Cu_Ka.A.value) of the components are set during m_tco.fit(). These (or some scaled version of the) values are what are exported with m_tco.get_intensities(). When you refit the background, you are "lifting" all the component gaussians up or down by a bit, but you are a) not changing their shape/area and hence b) not changing their .value.
The solution is to fix the background (if you are happy with it), and then refit the rest of the components again:
Thomas Aarholt
@thomasaarholt
# continuation of your code above
background = m_tco[0]
background.set_parameters_not_free()
m_tco.fit(bounded=False);

m_int_fit_tco = m_tco.get_lines_intensity(xray_lines_selected)
composition_fit_tco = si_TCO.quantification(m_int_fit_tco,method='CL',factors=kfactors, plot_result=True)
# you now get different printed output
OliDG
@OliDG
@thomasaarholt thanks for the thorough reply. First of all, you are correct, the background fitted values in the model changes dramatically, but you spot it, the line fitting (.A.value...) doesn't change at all.
I am so confused! I was convinced the background correction was applied during the get_line_intensity() instance, thus explaining this behaviour...it is even the way it is written in the documentation, first the model is fitted, then the background, then the line intensity are extracted (https://hyperspy.org/hyperspy-doc/current/user_guide/eds.html).
Does this mean that the subsequent background fitting is useless then? Should I change the code to 1/start fitting the multi-background then 2/ fixing its components then 3/ fitting the other components ? I also have a multifit part which takes 40 mn or so, I would therefore strongly avoid fitting the entire dataset several times.
Thomas Aarholt
@thomasaarholt
The docs are a little confusing, since there exist two separate .get_lines_intensity methods. The less advanced signal.get_lines_intensity() method takes a background window argument. The model.get_lines_intensity method is based off the regular model fitting algorithm.
You should:
Fit the background using the m.fit_background method, which also fixes the background so it won't change during calls to m.fit(). Then call m.fit(bounded=True) in order to fit the peaks without allowing the peaks to go upside down (negative intensity).
Thomas Aarholt
@thomasaarholt
image.png
The background fit does not look great though
OliDG
@OliDG
I have now changed and tested the code and starting with the background fitting does change the quantification results! thanks! But as you see yourself the fitting is not impressive, I have been trying to adjust the sigma windows, using multifit options, unfortunately it is not day and night.
The only fitting that was truly spot-on was using the bremsstrahlung background routine proposed by ZanettaPM (New EDS background #2836). It was amazing. However, it is only accessible in developper mode, and for some reasons the qt interface cannot work there, so it is not directly compatible with my existing code... it says it would be released in the next minor for a while now, but still not in the 1.7.1, anything I could do to help and speed it up ?
Thomas Aarholt
@thomasaarholt
I bet you're more than welcome to add to @ZanettaPM's PR, but in the meantime, I've rebased their branch on the latest hyperspy branch and pushed it to this branch: https://github.com/thomasaarholt/hyperspy/tree/bremsstrahlung_rebased
Try installing it from there - qt works with latest RELEASE_next_minor, so there is no reason it shouldn't work here.
9 replies
Wu Michael
@belleh1234567890_gitlab
Using the hs.plot.plot_signals to plot a model crashes every time. Has anyone experienced this?