Spectra Difference Generator

This is a tool for quickly aligning and comparing two (1D) data waves, with one being the reference to compare against and one is edited. The latter can be shifted, offset and scaled manually with respect to the reference to find the optimal overlap between the two or to extract certain spectral features from their difference etc. The main purpose is to calculate the difference (subtraction) between two data sets, but addition, multiplication and division are supported as well.

The underlying original data of both the edited wave and reference wave is never modified in any way. The comparison is handled with temporary data, and modifications (default wave ending "_mod") as well as the calculation result are always saved as new output waves. The default wave ending for the output is "_dif" for the difference, "_sum" for addition, "_mul" for multiplication and "_div" for division. All settings are saved in the output wave's notes and will be reloaded automatically with the next start. The program is fully folder aware.

I tried my best to test everything thoroughly, but you should verify the correct behavior yourself to be sure. Bug reports and suggestions for new features are always welcome.

Graphical User Interface (GUI) controls

To start the user interface, select one or two (1D) waves in the Data Browser and then go to 'Spectra Tools' -> 'Difference between Spectra ...' in the top menu. Or use the command line:

SpectraDifferenceGenerator(EditWave)
or
SpectraDifferenceGenerator(EditWave,refwave=ReferenceWave)

The second wave selected in the Data Browser (or the wave provided by the 'refwave' parameter) comes preselected as reference when starting the GUI. A quick overview of all controls can be found below:

  • The wave to modify and the reference wave are selected from the current folder. The current folder can be changed in the Data Browser during operation to use waves from a different folder.
  • Both the edited wave and reference wave can be smoothed (Gaussian broadened). The smoothing is saved in the output for the edited wave. The smoothing of the reference wave is only for display and to calculate the result, but is of course not actually applied to the original reference data. However, the smoothing setting is saved with the output and will be reapplied upon reloading.
  • Chose the mode of operation in the drop-down menu: A-B difference (- key), A+B sum (+ key), A*B multiplication (* key) or A/B division (/ key).
  • The result can be inverted (flipped in Y direction) with the Invert checkbox. The state of all checkboxes is saved with the output and will be reapplied upon reloading.
  • The edited wave is moved with the relevant controls of X/Y horizontal and vertical movement using the cursor buttons (cursor keys on the keyboard).
  • Scaling is done with the stretch (page up key) and shrink (page down key) controls.
  • The delta step of each modification can be set using the nearby variable controls.
  • Discard & Quit will just close the interface without saving anything.
  • Save & Quit will save the current state as output waves (edited wave name + "_mod" and "_dif" etc.) and then closes the interface. Optionally, the result is display in a new graph afterwards. At the same time all other graphs with the same data set displayed are updated as well to reflect the new modification settings.

Additional keyboard shortcuts: 

  • Hold Command (Mac) / Ctrl (Win) for changes in steps of 0.5 times the current delta value.
  • Hold Shift for changes in steps of 10 times the current delta value.
  • Hold both Shift + Command / Ctrl for changes in steps of 0.1 times the current delta value.
  • r key to ‘Reset All'
  • i key to toggle ‘Invert’ mode
  • p key to toggle ‘Pin Background’
  • Hold Command (Mac)/ Ctrl (Win) while pressing ‘Reset All’ and the reference wave will be set to none.
  • Hold Command (Mac)/ Ctrl (Win) while pressing ‘Save & Quit’ to override quitting the GUI. This can be used to compare many waves in quick succession.
  • Hold Option (Mac) / Alt (Win) while pressing ‘Save & Quit’ to force 'Use Ref. Name' for saving.

How the result is saved

The result is saved next to the input wave (the data to edit) as 'wavename +  "_mod"' for the modified data and  'wavename +  "_dif"' (or "_sum", "_mul", "_div" depending on the mode) for the (inverted) result. All modification settings are saved inside the wave note of both of these output waves. Here is an example where 'first spectrum' was edited and 'second spectrum' was the reference:

Loading the tool with the same initial data ('first spectrum' in this case) will reload the last session from the '_mod' or '_dif' wave (if '_mod' does not exist). Note that loading a '_dif' or '_mod' wave directly will just start a new session with this data (this makes it possible to create differences of differences etc.).The wave to edit and the reference wave can be in different folders. If neither a 'dif' (or 'sum', 'mul', 'div') or 'mod' wave with the same name is found in the input wave's folder, then the default settings are loaded. It is thus recommended to move the result and modified waves together with the original wave to a new folder to be able to reload the settings later. Then what will be reloaded if you have mixed result waves from summing, multiplying etc.? The loading hierarchy is 'mod' > 'dif' > 'sum' > 'mul' > 'div', and thus deleting the 'mod' and 'dif' waves will load the settings from the 'sum' wave if present. 

What is the purpose of the 'Use Ref. Name' setting?

Imagine the following scenario: A 'baseline' data and then several spectra have been measured in succession like this:

  1. baseline
  2. spectrum1
  3. spectrum2
  4. spectrum3
  5. ...

Now you want to remove the baseline wave from all the spectra, but the spectra should not be modified (for example, to preserve their relative shift or intensity). This is achieved by modifying (shifting, scaling etc.) the baseline for each spectrum individually and then saving the calculation result next to each spectrum. Load the baseline as wave to edit and one spectrum as reference and do the modification (since the roles of edited wave and reference are reversed here, this usually means 'Invert' needs to be activated). Then check 'Use Ref. Name' to save the result next to each spectrum. This will give:

  1. baseline
  2. baseline_mod
  3. spectrum1
  4. spectrum1_dif
  5. spectrum2
  6. spectrum2_dif
  7. spectrum3
  8. spectrum3_dif
  9. ...

What if you reload spectrum1 but have conflicting settings in waves spectrum1_mod and spectrum1_dif? In this case spectrum1_mod is loaded according to the hierarchy 'mod' > 'dif' > 'sum' > 'mul' > 'div'.

Settings inside the procedure header

The procedure header contains several settings as static constants, which can be edited to your liking:

static Constant kSaveAux = 0
static Constant kAuxMode = 0
static StrConstant kModEnd  = "_mod"
static StrConstant kOutEnd  = "_dif;_sum;_mul;_div;"
static StrConstant kAuxEnd  = "_smt;_int;"
// the following constants define the string keys for saving and loading parameters from wave notes
static StrConstant kScaleKey    = "Mod Scale"
static StrConstant kShiftKey    = "Mod Shift"
static StrConstant kOffsetKey   = "Mod Offset"
static StrConstant kModSmthKey  = "Mod Smooth"
static StrConstant kRefSmthKey  = "Ref Smooth"
static StrConstant kXCsrSetKey  = "Cursor Set"
static StrConstant kSetModeKey  = "Data difference;Data summed;Data multiplied;Data divided;"
static StrConstant kModeInvKey  = "Result inverted"
static StrConstant kPinBaseKey  = "Baseline scaled"
static StrConstant kRefNameKey  = "Reference name used"
  • The name endings ("_dif", "_mod" etc.) for saving the result waves can be modified here, should there be a conflict with other names in your experiment (in fact, Igor's own Differentiate function will also use 'dif' as name ending).
  • The string keys for saving the settings into the result wave's note are editable as well, but altering these keys might make saved results incompatible for reloading later (if these keys are changed again).
  • The GUI shows additional data derived from the calculation result, which is however normally not saved. Set kSaveAux to 1 to save this auxiliary data as well. The kAuxMode constant selects the type of data displayed (and saved). Currently two types of data exists: A smoothed version of the result (value = 0) and the integral of the result (value = 1).

Project Details

Current Project Release

Release File: Difference Generator_v4.32.zip
Version: IGOR.6.30.x-4.32
Version Date: Tue, 03/16/2021 - 09:05 pm
Version Major: 4
Version Patch Level: 32
OS Compatibility: Windows Mac-Intel
Release Notes:

Fixes:

  • Gauss convolution was not working well if data included NaNs.
  • Now the smoothing value is also displayed in the legend.
View All Releases

Nice looking, chozo! It has a lot of features that aren't in the Wave Arithmetic panel.

Thank you John! As the name suggests, this project started to make it possible to create differences with manual control and instant feedback for extracting even minute differences between measurements. But now that you mention it, it has indeed some things in common with the Wave Arithmetic package. This actually gives me the idea to add division and multiplication to the package for completeness.

EDIT: I have now updated the project to include four modes: Difference, sum, multiplication, and division.

I am seeing a pattern here: it seems that spectroscopists keep toolbags of useful packages that do similar things. I have a procedure ("wave scale and subtract") that does some of the things your panel can do, but as usual your panel is more sophisticated and does a lot more.

I will attach my little procedure here, just in case you are curious, but it doesn't do anything other than a small subset of what you do here.

ScaleSub106.zip

At one point in the mid- to late 90's (Igor 3 to 4), the toolbox was collected under the package SpXZeigR. I had an entire suite of features, help files, and tutorials. You won't find the original anymore (I have it somewhere on an archived CD). You will find a scattering of references to it, including a doi article that I did not even know about until just now.

I always wished to have had the time (and/or co-developers) to sustain the package.

Yes, I remember SpXZeigR, and I had that and other packages in mind when I wrote the comment. I had already developed a set of tools more specific to the techniques I was using at that time, so SpXZeigR never really entered my workflow.

RE SpXZeigR ... Sad as I am to see the package fade into the historical records, I am also glad to see a plethora of well-designed packages being provided to take its place.

I remember SpXZeigR. If tech support queries reflects popularity, it was fairly popular :)

John: The support that WaveMetrics provided to sustain SpXZeigR was greatly appreciated.

I didn't mean that it had lots of problems! No matter how well designed your package is, people will get into trouble. So any package that gets used a lot will generate some number of queries, regardless of its quality.

Tony, thank you for sharing your solution for this problem. It looks very nice and sufficient for the task (I would miss the shift in x, though). I hope the DiffGen project will be of use for you (and others) then. Yes, the problems and solutions of people in the community seem to be similar. Good thing, that there is a place to share. I am happy that I didn't have to come up with the ~10 projects I am using from the site (mostly your stuff, Tony). :)

By the way, I didn't know or use SpXZeigR. Must have been before my time. ;)

Tech Note 20 still exists in all its variants. And it still works!!! But the tools you folks have been making available are far superior.

Forum

Support

Gallery

Igor Pro 8

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More