Spectra Image Profile - Create profiles from, normalize and edit 2D / 3D spectral data

This project is used with 2D or (layers of) 3D data to create line profiles, and was inspired by the official Image Line Profile package (now replaced by the Line Profile function). The tool includes many convenient features tailored specifically to spectroscopic data (detector images, 2D scans or intensity maps etc.). Briefly, image data is integrated in horizontal or vertical direction within a region of interest (ROI, indicated by two lines). The resulting profile can be exported or used to modify the image data. Other features (such as normalization) are applied directly to the image data or layer. The tool modifies the input data directly, but a backup is saved upon start. Several convenient tools to work on 1D-3D data are included. The tool works directly on the input data, but a backup is saved upon start.

I tried my best to test everything thoroughly, but users should verify the correct behavior for critical applications. Bug reports and suggestions for new features are always welcome. Special thanks to Oleg Dyachok from Uppsala University for suggesting and testing improvements for version 4.0 and beyond.

User Interface controls

Select a 2D or 3D wave in the Data Browser and then go to ‘Spectra Tools’ → ‘Image Profile …’ in the main menu or use the command line:

Below is an overview of all controls (here, the case of a 3D input is shown):

First, the tool prints a record of all actions to the history window. This can be switched off by unchecking Print to History.

Integration range (Region-of-Interest or ROI) controls

• Choose the integration mode: horizontal (h key) or vertical (v key) direction. Diagonal or free-form modes (like in the Line Profile tool) are not available.
• ROI lines can be adjusted via the Pos. and Size values, by dragging the lines or by using the cursor keys on the keyboard.
  • When using the cursor keys, hold the shift key to speed up changes by 4x. Hold the ctrl / cmd key to slows down changes by 1/2. Holding both the ctrl / cmd + shift keys slows changes down to one delta unit (one row / column).
• Snap to Grid: The ROI lines are constrained to include only full rows / columns.
• Switch to full or min. ROI range: activate the respective check box or use the page up / page down key, respectively.

Controls related to the integrated profile

• Norm to 1: The profile (not the image) will be normalized to fall between 0 and 1.
• Mouse Csr: An additional profile is plotted for comparison when the mouse cursor is moved across the image, which shows the current row or column under the mouse cursor (toggle with the m key). This profile is not saved.
• Save button: Saves the current profile in the folder of the input with a name-end tag depending on the selected range:
  • The end tag is of the format “_H(xxx-xxx)” (horizontal mode) or “_V(xxx-xxx)” (vertical mode), where xxx is the start and end of the ROI range, e.g., “mydata_H(16.7-19.3)”. If the full image is selected the tag is “_prH” or “_prV”, respectively.
  • Hold the shift key to force the use of “_prH” / “_prV” regardless of the selected range. The ROI position and size setting is always saved inside the output's wave note.
  • Hold the alt key to save the output into the source wave's folder instead of the current folder.
• Activate the label axis checkbox to export additional axis information for plotting the output profile. This axis information is generated from the dimension labels of the source (if available) and exported as user-tick waves (with endings '_axV' and '_axT' for the axis value and label waves, respectively).
• Snapshot button: Creates and displays a backup of the current profile for comparison (this snapshot will not be saved). Each button press replaces this snapshot with the current profile. Hold ctrl to remove the current snapshot instead.
• Load in drop-down: Load an external wave (1D) for comparison with the current profile. This data can also be used for normalization (division), subtraction, and alignment.

Profile post-processing options

Several options for post-processing applied to the profile are selectable via a drop-down menu. Post-processing will be applied live while the profile is changing, and is also saved in the output. It may be useful to apply post-processing before using other features such as division or subtraction. Available methods are: (simple) smoothing, Gaussian smoothing (via convolution), calculation of the integral and the derivative, and fast-Fourier transformation (FFT). FFT is not applied to the profile itself and is instead displayed in a subgraph. The FFT result is also exported using the end tag '_pp'.

Changes to the 2D / 3D data

• Accept & Quit: Closes the UI while keeping all edits made to the 2D/3D data so far.
• Undo All: Resets all changes made to the 2D/3D data (returns to the condition at start).
• Undo Last: Reverts the last change made to the 2D/3D data. This will only work once, i.e., you cannot undo multiple steps.
• Save as: Saves the 2D/3D data under a different name and undoes all edits made to the initial data.
• Quit: Closes the window undoes all edits made to the 2D/3D data.

Normalization

Normalization works only on the current layer of 3D data. The following normalization methods are available from the drop-down list:

• Divide by Profile: This divides the image / layer by the current or loaded profile (each column in horizontal / each row in vertical mode, respectively). Any selected profile post-processing is applied before this step, i.e., the profile as displayed in the main graph is used.
• Divide by Max. or Area: Finds the maximum or area of each row (horizontal mode) or column (vertical mode) and divides the column / row by this value. This is useful, for example, for equalizing data along one direction (use the Keep Int. checkbox to prevent downscaling of the data to 1).
• A user-defined function (this item is only displayed for compatible data): Normalize by user provided information. This can be any data parameter or wave saved in the current experiment. Examples are normalization by a secondary readout, time, concentration etc. See further below on how to set up your own normalization function.

Options:

• Smooth: The data / profile is smoothed before normalization is applied, which reduces the effects of noise in the input. Hold the ctrl / cmd key while selecting a normalization option to skip smoothing or set the value to 0.
• Keep Int. checkbox: This will preserve the average intensity after normalization, i.e., the data is scaled back to the initial, average intensity level.

The Delete Selected button

This deletes columns (horizontal) or rows (vertical) within the current ROI. If lines at the wave's edge are removed then the axis scaling is adjusted to account for the deleted lines. If lines in the center are removed then the scaling is not adjusted (wave scaling is linear and continuous and cannot account for missing points in-between). Be aware that this may break the relation between data and axis values. If the axis scaling is important (e.g., for a continuous energy scale), either refrain from using this feature or take care to only remove lines at the edges of the data. In case of 3D data, this feature will delete columns / rows on all layers.

The Subtract button

The current (or loaded) profile is subtracted from each column (horizontal mode) or row (vertical mode) of the image / layer. This can be particularly useful for removing fixed spectral features across a scanned map. The scale (x) value can be adjusted to pre-scale the intensity of the profile before subtraction.

An additional function is available in horizontal mode only: the profile can be moved in x relative to the y-axis. Briefly, the y-axis value is added to (or subtracted from) the profile’s x scaling each step, which leads to a shifting profile in diagonal direction. This is useful to subtract features moving with increasing values on the y axis, such as peaks shifting with increasing photon energy.

• Hold ctrl / cmd to subtract while shifting towards increasing Y values.
• Hold alt / option to subtract while shifting towards decreasing Y values.

Below example shows a diagonal subtraction result (here using ctrl):

The Shift +/- Y buttons

This is limited to the horizontal direction (regardless of the selected mode). In case of 3D data, all layers are processed. This feature shifts (horizontal) columns, where the x-scaling of each column is modified by adding or subtracting the respective y-axis value; in result, the image is transformed into a parallelogram. This will straighten features in the data which were shifting with y. An example would be a photon-energy (y) scan of a spectrum where certain spectral features increase in their (x) emission energy: Subtracting the photon-energy scale from a photoelectron spectrum in kinetic energy yields binding energy. The effect is illustrated again with above example data:

Hold ctrl / cmd to automatically delete any partially filled data after the shifting was done (the parallelogram is 'cut' down into a rectangle). Note that this might fail if there is not even one complete row of data left (i.e., the parallelogram's skew is too large to fit a rectangle inside).

Align Features functionality

This tool searches for similar features in each column (horizontal mode) or row (vertical mode) of the image / layer and aims to align these features for maximum overlap by shifting each column / row by a certain amount. The algorithm uses wave correlation and shifts the data in whole steps (no interpolation) or, when the Sub-steps checkbox is active, sub-steps (interpolated). It is useful, for example, to correct slight shifts of peaks in a spectrum or to estimate the shift of a peak versus some other parameter like time or position:

The algorithm uses the first column (bottom to top in horizontal mode) or row (left to right in vertical mode) of the data as reference and tries to align all other columns / rows to this reference. Hold shift to use the last row / column instead (top to bottom in horizontal mode and right to left in vertical mode). If you want to align to the calculated or loaded profile instead, hold the alt / option key while pressing the Align Features button. A menu will appear which lets you select the desired reference.

Often, it is undesirable to use the full range of the data for alignment. By placing the A and/or B cursors onto the image you can limit the algorithm to the region between the cursors to, e.g., selectively align only a single feature of interest contained.

• Activate Smooth to smooth the data before aligning. This can improve the results for noisy data.
• Activate Edges to align for a maximum overlap of edge features instead of peak features (using the data's derivative).
• Substeps aligns in steps smaller than the data-point delta by using interpolation (default). Uncheck to shift in integer steps.
• Some data points will be outside the input range when shifting the data. Usually, these out-of-bounds data points remain unset (NaN). Activating Fill will instead repeat the end points of the data to fill this region.
• Activate Output to export found shift values into a wave with the name ending '_shft'. The image's wave scaling is applied.

Note that the algorithm is ‘dumb’ in the sense that it only looks for maximum correlation and does not judge whether 'peak features' really match etc. In the worst case this just aligns artifacts such as noise, so use with caution.

Image contrast and color controls

Here you can choose the used color table and adjust the contrast of the image / layer: select the desired color table by using the drop-down menu and move the Min and Max adjustment sliders until the contrast is as desired. The sliders are limited to the maximum and minimum data values of the image or current layer, so the contrast cannot be reduced only increased. If you reverse the Min / Max slider positions the colors of the image will be inverted. Activate Log Scale to switch to a logarithmic scale for both the image and profile. Switching to this mode will not work if the data is completely in the negative range or zero.

Additional controls for 3D data (volumes)

Layer and Merge controls

The Layer value selects the currently displayed layer of a 3D data set. Profiles are calculated from, and most functions for editing data will only act on, this one layer (there are exceptions like Delete Selection and Shift +/-). The Delete button deletes the current layer. The Merge All button will merge all (remaining) layers into a 2D image which can then be edited further. All changes can be reverted by using the Undo buttons. The scaled position of the current layer is shown as an annotation at the top right in the image.

Displaying profiles from all layers

Activating Multiple Profiles will simultaneously plot profiles of all layers, which are generated from the current ROI, for comparison with the profile of the current layer (shown in bold). The profiles will also be exported into a wave ending with '_map'. This is useful, e.g., to track small changes across many layers. The Separation value controls the vertical spacing between traces in the profile plot in percent of the (global) maximum y value. The maximum number of profiles (layers) which can be plotted is 200. Beyond this number the profiles are saved but are not shown.

Clicking Display Z-Profile will open an inset graph which plots the change of intensity in z- (i.e., layer-) direction, useful for tracking signal variations across layers. The type of information can be changed via the drop-down menu: average, maximum or minimum intensity.

Rotating a 3D volume

3D data can be rotated in 90-degree steps around the principal axes x,y,z in clockwise and counter-clockwise direction using the designated buttons at the bottom of the panel. After rotation the first layer is displayed by default, but if empty (zeros throughout) then the image might look all white (this is no bug). Rotation might reverse the x- and/or y-axes, depending on the new orientation of the volume's contents. Thus, the data might look 'reversed' when displayed in a normal image plot.

Quick access functions in the menu

Some useful functions are available in the Spectra Tools menu and the right-click menu of the Data Browser:

• Append 1D,2D Datasets into 2D Data: Combines columns (y dimension) of multiple datasets into one consecutive 2D wave. The x dimension is extended to cover the full x range of all datasets if needed.
• Stack 2D Datasets into 3D Data: Stacks multiple 2D spectra (e.g., MCP images) to produce a 3D array.
• Profile of 2D\3D Data: Same as creating a profile over the full range using the UI. This will yield 1D data for a 2D input and 2D data or a 3D input. Hold shift while invoking this menu item to define a sub-range in the respective dimension.
• 2D Image from 3D Data: Sums 3D data in the layer (array) direction to output a 2D image.
• Split Up 2D\3D Data: Separates 2D / 3D data into individual waves holding columns, rows or layers, respectively. The individual waves are saved in a folder named after the input.

In Igor 9, these options are also available in the right-click menu of the data browser.

Creating profiles, splitting or combining data from the command line

Some quick access functions can be directly invoked. To create a profile use:

Here the mode is an integer between 0 and 2, with the meaning of 0 = vertical profile (average rows), 1 = horizontal profile (average columns), 2 = z profile (average layers). Use the optional 'from', 'to' or the 'from_Point', 'to_Point' parameters to define the integration sub-range in terms of scaled values or point values, respectively. If both are specified the former takes precedence.

The data can be split along one dimension via:

Here, the dimension parameter has the same meaning as the mode above.

To reduce the fidelity of data you can bin data points by using:

Here, binX, binY, binZ must be integer values ≥1, which sets by how many points the data in the respective dimension is binned. For example, using binX = 2 will result in a wave with half of the points in the x (row) dimension (binning 2 points into 1). The input data can be binned in multiple dimensions simultaneously.

To combine data in the x- (rows), y- (column) or z- (layer) directions, use:

The dimension parameter has the same meaning as above. wave_list is a string list of waves (use full paths if the waves are not in the current folder).

Procedure settings and user button code

The user-defined (normalization) button

Two functions and one global constant are used to provide the functionality for the user button:

• kUserInfoBtnName: Defines the menu item's description and can be dynamically changed to reflect the purpose of the function. This name should be short to fit inside the menu.
• Profile_ActivateUserInfoButton(inputWave): A function for deciding whether conditions for normalization are fulfilled. A return value of 1 activates, any other value disables user normalization. One could for example check here if the required information is contained in the 2D data or loaded in the current experiment.
• Profile_ExecuteUserInfoButton(inputWave): This function is called to do the normalization (or whatever you like to do here). Return a scaling factor for scaling the intensity to its average after normalization (returning 0 skips this step).

While this feature is intended for normalization, it is really a free-for-all function for any modification to the 2D data you may want. You could instead write a function which, for example, splits the data in half or applies interpolation or does something completely different in each experiment file.

You may modify the global constant and these two functions at the beginning of the procedure file directly, but it is more convenient to provide your own code in a separate file or inside the experiment environment. Use the Override keyword in front of the Function / StrConstant statement to override the standard definitions in your own implementation. You may use the project’s procedure code as basis for writing your own version. In the package, the function's purpose is to normalize to the ring current of synchrotron experiments as extracted from the 2D data as shown here:

Static StrConstant kUserInfoBtnName = "Ring Current"    // the title of the user button
 
Static Function Profile_ActivateUserInfoButton(Wave inwave)
    Variable activate = 0
    // +++ conditions for activation +++
    activate = StringMatch(GetDimLabel(inwave,1,0),"*mA*")  // check if dim-labels contain "mA" values
    activate = StringMatch(note(inwave),"*ring current normalized*")? 0 : activate  // already normalized?
    // +++
    return activate // return 1 to activate button
End
 
Static Function Profile_ExecuteUserInfoButton(Wave inwave)
    Variable valAvg = 0
    // +++ ring current normalization procedure +++
    Variable i, value
    for (i = 0; i < DimSize(inwave,1); i += 1)
        String Currlabel = GetDimLabel(inwave,1,i)  // get the information from the column label
        sscanf Currlabel, "%s (%f mA)", Currlabel, value    // ring current info as "time (XXX mA)"
        if (value != 0 && !numtype(value))
            inwave[][i] /= value
        endif
        valAvg += value // add the values for intensity re-normalization later
    endfor
    valAvg /= i+1
    Note inwave, "ring current normalized"  // write into note that the normalization has been done
    // +++
    return valAvg   // return the intensity normalization factor
End