Spectra Background Remover - Subtract common spectroscopic background functions
This project provides functionality for calculating and removing backgrounds from (1D) spectra. The intention is to correct inelastic scattering backgrounds from photoemission spectra, but other, more general background types are included as well. Both a user interface for interactively subtracting backgrounds and a user function for calculating backgrounds from input data is provided.
Note that it is advised to rather use background functions in curve fitting instead of subtracting data 'by hand' for analysis. Most of the background types provided in this tool are also found as fit functions in the Multipeak Fit package (version 3) for Igor 9.
I tried my best to test everything thoroughly, but you should verify the correct behavior yourself for critical applications. Bug reports and suggestions for new features are welcome.
Basic controls in the user interface (UI):
Select a (1D) wave in the Data Browser and then navigate to 'Spectra Tools' → 'Background Subtraction ...' in the main menu or use the command line:
BackgroundSubtractGUI(mydata)
Controls:
- Type: Select the desired background type.
- Avg. Range: The data intensity around the cursor positions is used as input for anchoring the background. Here, the data is averaged over the set range to smooth out noise (a value of 0 will switch off averaging).
- Left & Right Cursor: Sets the cursor positions, which indicate the effective range for background calculation. Cursors can also be dragged directly by hand.
- Export Background: Exports the background (blue; name ending "_bck") and the subtracted data (green; ending "_net") into the folder of the input.
- Plot after Export: Creates a separate plot of the exported data.
- Subtract: Subtracts the current background (blue) from the data and then continues with the subtracted data. You can successively subtract multiple types of backgrounds one after another.
- Undo All: Reverts to the initial data.
- Overwrite & Quit: Overwrites the input with the current result (red trace). Will not have any effect if nothing was subtracted.
- Background-specific controls will appear for each background type, and their function is explained via tool-tips when hovering the mouse cursor above each control element.
Included background types:
- Constant: Subtract a constant value.
- Step: Smooth step function which connects the two marked endpoints (this uses tanh(x) as the step function).
- Linear: Linear function through the two endpoints.
- Exponential: Normalized exponential function connecting the two endpoints.
- Double Exponential: Double exponential function fitted within the selected range.
- Polynomial: Polynomial function fitted within the selected range.
- Total Sum: The total sum inelastic scattering background after X. Li et al., J.Electron Spectrosc. Relat. Phenom. 63 (1993) 253-265
- Shirley: The iterative Shirley inelastic scattering background after A. Proctor and P. A. Sherwood, Anal. Chem. 54, 13-19 (1982)
- Tougaard: Three- and four-parameter universal scattering background after S. Tougaard, Surf. Interface Anal. 25: 137-154 (1997)
- Tougaard Data: Scattering background using scattering cross-section data after S. Tougaard, Surf. Interface Anal. 25: 137-154 (1997).
Loaded data should have a positive X scale starting from zero and at least the same total energy range as the input spectrum, like this one:
(data not starting from x = 0 will be padded with zeros, and data which is too short may give the wrong result)
Running your own code:
You can generate backgrounds using the function:
GenerateBackground(backstruct, datawave, backwave, [from, to])
Populate the BackRemoveStruct structure with the desired parameters and provide both the original data (datawave) and a wave which the background will be written to (backwave). Both datawave and backwave need to have the same size and scaling. The 'from' and 'to' parameters optionally set the start and end positions of the background calculation using the wave's x scaling (the first and last points of the input data are used if omitted). Here is an example:
Function backgroundtest(Wave inwave) STRUCT BackRemoveStruct s // the structure contains all relevant parameters BackRemoveInitialize(s) // initialize s.Functype="Shirley" // we use the Shriley background Duplicate/O inwave, $(NameOfWave(inwave)+"_bck")/Wave=backwave String statusmsg = GenerateBackground(s, inwave, backwave) Print statusmsg End
The BackRemoveStruct structure looks like this:
Structure BackRemoveStruct String FuncType // Type: Constant, Step, Linear, Exponential, Polynomial, Total Sum, Shirley, Tougaard, Tougaard Data Variable avgPercent // y Values around range ends positions get averaged // function specific: Variable expScale // Exponential Variable polyDegree // Poly Variable fullLinear // Line // for Step Variable stepPos // x center position Variable stepSmooth // smoothing value // for Tougaard (universal CS parameters) Variable TouB Variable TouC Variable TouD Variable TouT Variable TouSmooth // insulator step (T0) smoothing (should not be 0) // Tougaard Data Variable CSscale // scaling (default 1) String CSdata // name of CS wave in current folder // Shirley and Total Sum Variable followData // follow data outside range Variable smoothData // integer between 3 and 20 EndStructure
Project Details
Current Project Release
| Release File: | Background Remover_v1.56.zip (12.28 KB) |
| Version: | IGOR.8.00.x-1.56 |
| Version Date: | |
| Version Major: | 1 |
| Version Patch Level: | 56 |
| OS Compatibility: | Windows Mac-Intel |
| Release Notes: |
|
Forum
Support
Gallery
Igor Pro 9
Learn More
Igor XOP Toolkit
Learn More
Igor NIDAQ Tools MX
Learn More
Very nice, Stephan. For years I have been meaning to add Shirley and Tougaard to the baselines package, but the only inelastic scattering stuff I do is Raman, no XPS or anything like that, so I never got around to it. This is great.
January 23, 2021 at 02:19 am - Permalink
This would make a more graceful exit if the data browser window is closed:
January 23, 2021 at 02:39 am - Permalink
For the buttons in the top left corner of the window to display properly on a mac, without text clipping, the minimum widths are about 63 and 114. Mac uses a larger font than windows for buttons (and doesn't allow you any say in the matter).
January 24, 2021 at 03:20 am - Permalink
Tony, thank you very much for your kind comments. Getting useful comments like this is one of the reasons to finally release this tool here! In fact, I did not realize the problem with the data browser, since I cannot imagine running Igor without. ;) Also, thanks for the size check on Mac. I know about the size mismatch between the two OSs too well, but unfortunately don't have easy access to a Mac. I can 'lend' one from time to time, and will try to test my GUI design. I uploaded a small patch to address these points.
I was using your baseline package in the past and found it very useful. Feel free to take any code from here if you find it useful to add to your package.
January 24, 2021 at 06:15 pm - Permalink
Hi Chozo
I just thought I would say that this, along with your spectra difference script, will be extremely useful for so many people using Igor in the spectroscopy community. The work you have done on the multipeak package is equally awesome. It is all much appreciated, and it makes it much easier to recommend Igor as a solution that just works "out the box" for this type of analysis.
All the best,
Rob
February 9, 2021 at 06:25 am - Permalink
Hi Robert,
Thank you very much for your kind comment! I am super happy that these tools are of use for others. Did you already had the chance to test the new Multipeak Fit package? Just let me know if you find any bugs or have suggestions for improvements. I am currently polishing up a third project for release, which is inspired by the Image Profile code to work on 2D images (ARPES data, MCP images, energy scans etc.). I hope to get it out next week.
Best wishes,
Stephan
February 10, 2021 at 02:50 am - Permalink
In reply to Hi Robert, Thank you very… by chozo
No problem. I have been using the new multipeak fit package. It is great and has already found it's way into my workflow and replacing my notably worse scripts for fitting voigt profiles. An easy way of fitting multiple Voigt functions with the two widths defined (rather than a shape parameter) is critical for so many of us and has really been lacking in Igor until now.
I have not come across any bugs yet... but I will let you know! My only long-term comment would be setting advanced constraints is still somewhat unintuitive. Even adding some more check boxes to quikcly allow coefficients above K1 to all be set the same would be a time saver.
Sorry for taking this thread off-topic! To bring it back on-topic, in case you didn't know, there is a problem installing the background remover project using the Project Updater. You get the error message: "Could not find packages at https://www.wavemetrics.com/project-releases/21532."
Thanks again
February 18, 2021 at 02:13 am - Permalink
Thank you for the report. I have fixed the project page now and the Updater should happily accept the package.
OK, I will go a bit off-topic as well: If you have ideas or requests for the new Multipeak Fit then you should definitely write an email to the support. The WaveMetrics staff can otherwise not imagine what people want exactly, and I too have often just proposed / implemented what I need or what was on the request-list anyway. I think it would be a matter of half an hour to implement a checkbox for the height / area coefficient K2. For higher coefficients it may be a bit trickier, since not all peaks have them. Anyway, just let your wish-list be heard. :)
February 18, 2021 at 05:24 pm - Permalink
What was it that prevented updater from installing the file? I worry that the code that parses the 'all releases' web page is a bit fragile. In fact, I'm tempted to change the installer and updater to use the latest release info from the main project page and ignore the all-releases page.
February 18, 2021 at 11:51 pm - Permalink
Tony, I forgot to set the 'Version' field, which apparently makes the Updater think there is no release. I also found a small bug with the right-click menu, but I post that below the Updater project.
February 19, 2021 at 12:52 am - Permalink
Hi Chozo,
I am about to release a data analysis package for our endstation. I chose gitlab so users can always get the latest version easily. The software will be open and free to use, modify and distribute.
I was thinking of using your .ipf, it's a great piece of code. I will have to change a couple of lines and launch the BackgroundSubtractGUI from the TracePopup Menu, and maybe a couple more minor changes that will not affect the main functionality. Changes will be indicated with extra comments. All your comments will be left intact and I will add a link to the current thread for interested users.
Am I allowed to do it?
December 14, 2022 at 08:02 am - Permalink
Hi EG,
Thanks for your message. I am happy that you like my project. Sure, the code here on WM.com is for everyone to use and reuse from my perspective. Feel free to include this in your project as you described. Also, if you made some neat improvements to the code, let me know and I can update the project here eventually. I would also be interested in having a look at your package when it is released. Maybe I can learn something from it as well.
December 14, 2022 at 04:38 pm - Permalink
Hi Chozo,
I did not change something essential in your code, it works great. Only minor changes to adapt to out Packages DB. You can have a look at our git repository. The package is under heavy development and we need to add some documentation, hopefully soon. Your code is in the Analysis/ folder.
Thank you very much.
March 15, 2023 at 02:02 pm - Permalink
My packages supports menu keyboard shortcuts which are loaded from a settings file. If you want to add keyboard shortcuts to common menu entries, place the content (Spectra Tools settings.dat) of the attached zip file in the same folder as the package procedure and edit as necessary.
July 27, 2023 at 11:54 pm - Permalink