Spectra Background Remover - Subtract common spectroscopic background functions

This project provides functionality for calculating and removing (subtracting) backgrounds from (1D) data. The main purpose is to correct inelastic scattering backgrounds from photoemission spectra, but some general background types for other purposes are included as well. Both an user interface for interactively subtracting backgrounds as well as a function for calculating backgrounds from input data is provided.

Note that it is always better, if possible, to directly include the correct background function into curve fitting directly instead of subtracting data 'by hand' for analysis. Most of the background types provided in this tool are also found as fit functions in the Multipeak Fit package (version 3) for Igor 9.

I tried my best to test everything thoroughly, but you should verify the correct behavior yourself for critical applications. Bug reports and suggestions for new features are welcome.

Basic controls in the user interface:

To start the user interface, select a (1D) wave in the Data Browser and then select  'Spectra Tools' -> 'Background Subtraction ...' in the top menu. Or use the command line:

BackgroundSubtractGUI(mydata)

  • Type: Select the desired background function here.
  • Avg. Range: The range is marked with the cursors and the Y height of the current data is used as start- and endpoints. The data is averaged over a certain range to smooth out noise. Set to zero if the range should end at the exact cursor location
  • Left & Right Cursor: Shows and modifies the cursor positions, which mark the effective range for the background calculation. The cursors can be dragged directly in the graph as well.
  • Export Background: Saves the current background (blue trace; name ending "_bck") and current preview of the subtracted data (green trace; ending "_net") next to the input data.
  • Plot After Export: Activate to create a separate plot of the exported data when pressing the Export Background button.
  • Subtract: Subtracts the current background (blue trace) from the data and then works with the subtracted data. This can be used to successively subtract multiple types of backgrounds one after another.
  • Undo All: Reverts to the initial data.
  • Overwrite & Quit: Overwrites the original data with the current spectrum (red trace). This will not have any effect on the original data if no background was subtracted in the first place.
  • Background-specific controls will appear for each background function, which are explained in tooltips.

Included background types:

  • Constant: Subtract a constant value.
  • Step: Smooth step function which connects the two endpoints (using tanh()).
  • Linear: Linear function through the two endpoints.
  • Exponential: Normalized exponential function connecting the two endpoints.
  • Double Exponential: Double exponential function fitted within the selected range.
  • Polynomial: Polynomial function fitted within the selected range.
  • Total Sum: The total sum inelastic scattering background after X. Li et al., J.Electron Spectrosc. Relat. Phenom. 63 (1993) 253-265
  • Shirley: The iterative Shirley inelastic scattering background after A. Proctor and P. A. Sherwood, Anal. Chem. 54, 13-19 (1982)
  • Tougaard: Three- and four-parameter universal scattering background after S. Tougaard, Surf. Interface Anal. 25: 137-154 (1997)
  • Tougaard Data: Scattering background using scattering cross-section data after S. Tougaard, Surf. Interface Anal. 25: 137-154 (1997).
    This data should have a positive X scale starting from zero and at least the same total energy range as the input spectrum, like this one:
    (data not starting from x = 0 will be padded with zeros, and data which is too short may give the wrong result)

Running your own code:

You can generate backgrounds using the function:

GenerateBackground(backstruct, datawave, backwave, [from, to])

Populate the BackRemoveStruct structure with the desired parameters and provide both the original data (datawave) and a wave which the background will be written to (backwave). Both datawave and backwave need to have the same size and scaling. The 'from' and 'to' optional parameters are for setting the start and end positions of the background calculation in the wave's x scaling. If these parameters are omitted then the first and last points of the input data are used. Here is an example:

Function backgroundtest(Wave inwave)
    STRUCT BackRemoveStruct s       // the structure contains all relevant parameters
    BackRemoveInitialize(s)         // initialize
    s.Functype="Shirley"            // we use the Shriley background
    Duplicate/O inwave, $(NameOfWave(inwave)+"_bck")/Wave=backwave
    String statusmsg = GenerateBackground(s, inwave, backwave)
    Print statusmsg
End

The BackRemoveStruct structure looks like this:

Structure BackRemoveStruct
    String FuncType     // Constant, Step, Linear, Exponential, Polynomial, Total Sum, Shirley, Tougaard, Tougaard Data
    Variable avgPercent // y Values around range ends positions get averaged
    // function specific:
    Variable expScale   // Exponential
    Variable polyDegree // Poly
    Variable fullLinear // Line
    // for Step
    Variable stepPos    // x center position
    Variable stepSmooth // smoothing value
    // for Tougaard (universal CS parameters)
    Variable TouB
    Variable TouC
    Variable TouD
    Variable TouT
    Variable TouSmooth  // how much the insulator step for parameter T0 will be smoothed (should not be 0)
    // Tougaard Data
    Variable CSscale    // scaling (default 1)
    String CSdata       // name of CS wave in current folder
    // Shirley and Total Sum
    Variable followData // follow data outside range
    Variable smoothData // integer between 3 and 20
EndStructure

 

Project Details

Current Project Release

Release File: Background Remover_v1.55.zip
Version: IGOR.6.30.x-1.55
Version Date: Sat, 03/18/2023 - 03:11 pm
Version Major: 1
Version Patch Level: 55
OS Compatibility: Windows Mac-Intel
Release Notes:
  • Added double exponential function.
  • Rearranged controls for clarity.
View All Releases

Very nice, Stephan. For years I have been meaning to add Shirley and Tougaard to the baselines package, but the only inelastic scattering stuff I do is Raman, no XPS or anything like that, so I never got around to it. This is great.

This would make a more graceful exit if the data browser window is closed:

Menu "Spectra Tools"
    "Background Subtraction ...",/Q, CreateBrowser;BackgroundSubtractGUI($GetBrowserSelection(0))
End

 

For the buttons in the top left corner of the window to display properly on a mac, without text clipping, the minimum widths are about 63 and 114. Mac uses a larger font than windows for buttons (and doesn't allow you any say in the matter).

Tony, thank you very much for your kind comments. Getting useful comments like this is one of the reasons to finally release this tool here! In fact, I did not realize the problem with the data browser, since I cannot imagine running Igor without. ;) Also, thanks for the size check on Mac. I know about the size mismatch between the two OSs too well, but unfortunately don't have easy access to a Mac. I can 'lend' one from time to time, and will try to test my GUI design. I uploaded a small patch to address these points.

I was using your baseline package in the past and found it very useful. Feel free to take any code from here if you find it useful to add to your package.

Hi Chozo

I just thought I would say that this, along with your spectra difference script, will be extremely useful for so many people using Igor in the spectroscopy community. The work you have done on the multipeak package is equally awesome. It is all much appreciated, and it makes it much easier to recommend Igor as a solution that just works "out the box" for this type of analysis.

All the best,

Rob

Hi Robert,

Thank you very much for your kind comment! I am super happy that these tools are of use for others. Did you already had the chance to test the new Multipeak Fit package? Just let me know if you find any bugs or have suggestions for improvements. I am currently polishing up a third project for release, which is inspired by the Image Profile code to work on 2D images (ARPES data, MCP images, energy scans etc.). I hope to get it out next week.

Best wishes,

Stephan

In reply to by chozo

No problem. I have been using the new multipeak fit package. It is great and has already found it's way into my workflow and replacing my notably worse scripts for fitting voigt profiles. An easy way of fitting multiple Voigt functions with the two widths defined (rather than a shape parameter) is critical for so many of us and has really been lacking in Igor until now.

I have not come across any bugs yet... but I will let you know! My only long-term comment would be setting advanced constraints is still somewhat unintuitive. Even adding some more check boxes to quikcly allow coefficients above K1 to all be set the same would be a time saver.

Sorry for taking this thread off-topic! To bring it back on-topic, in case you didn't know, there is a problem installing the background remover project using the Project Updater. You get the error message: "Could not find packages at https://www.wavemetrics.com/project-releases/21532."

Thanks again

Thank you for the report. I have fixed the project page now and the Updater should happily accept the package.

OK, I will go a bit off-topic as well: If you have ideas or requests for the new Multipeak Fit then you should definitely write an email to the support. The WaveMetrics staff can otherwise not imagine what people want exactly, and I too have often just proposed / implemented what I need or what was on the request-list anyway. I think it would be a matter of half an hour to implement a checkbox for the height / area coefficient K2. For higher coefficients it may be a bit trickier, since not all peaks have them. Anyway, just let your wish-list be heard. :)

What was it that prevented updater from installing the file? I worry that the code that parses the 'all releases' web page is a bit fragile. In fact, I'm tempted to change the installer and updater to use the latest release info from the main project page and ignore the all-releases page.

Tony, I forgot to set the 'Version' field, which apparently makes the Updater think there is no release. I also found a small bug with the right-click menu, but I post that below the Updater project.

Hi Chozo,

I am about to release a data analysis package for our endstation. I chose gitlab so users can always get the latest version easily. The software will be open and free to use, modify and distribute.

I was thinking of using your .ipf, it's a great piece of code. I will have to change a couple of lines and launch the BackgroundSubtractGUI from the TracePopup Menu, and maybe a couple more minor changes that will not affect the main functionality. Changes will be indicated with extra comments. All your comments will be left intact and I will add a link to the current thread for interested users.

Am I allowed to do it?

 

Hi EG,

Thanks for your message. I am happy that you like my project. Sure, the code here on WM.com is for everyone to use and reuse from my perspective. Feel free to include this in your project as you described. Also, if you made some neat improvements to the code, let me know and I can update the project here eventually. I would also be interested in having a look at your package when it is released. Maybe I can learn something from it as well.

 

 

Hi Chozo,

I did not change something essential in your code, it works great. Only minor changes to adapt to out Packages DB. You can have a look at our git repository. The package is under heavy development and we need to add some documentation, hopefully soon. Your code is in the Analysis/ folder.

Thank you very much.

 

My packages supports menu keyboard shortcuts which are loaded from a settings file. If you want to add keyboard shortcuts to common menu entries, place the content (Spectra Tools settings.dat) of the attached zip file in the same folder as the package procedure and edit as necessary.

Spectra Tools settings.zip

Forum

Support

Gallery

Igor Pro 9

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More