Simple operation - batch operation of waves. How to?
MartinG
Fri, 10/07/2016 - 07:24 am
I am new to Igor and in general to programming. I believe my operation is quite simple, but can't figure out how to do it.
I have a series of waves - 97 in total.
Wave one contains the x-axis information (temperature) for the 96 y-axis information (signal) waves.
Basically it is 96 "signal vs temperature" data sets with identical scaling.
I would like to perform the same operation on all (y-)waves:
1) Smooth
2) Differentiate
3) wavestats to extract min, max, etc
4) Calculate "Area under the curve"
4) save relevant Wavestats results
I know how to do this on a single set of (x,y) waves. I might not be pretty, but it works ;-)
(I have pasted code below - in this code "A1" is the y-wave and "ScanTemp" is the x-wave)
Now the problem! I wouldl like to loop this calculation for wave A1, A2, A3,......H12. I can make a loop that reads out the wave names, and my thought was that if I could make a string with content A1, then I could use this as the name of the wave to be analysed; e.g. first I make a string with A1 then tell calculation to be done on wave A1 (by referencing to the string), then in next loop the string is changed to A2 and the calculation is run again. Hope it makes sense.
I also considered using rename and make a temporary wave for the calculation loop. But that I cannot make work either.
How do I do this in an easy to implement (simple code) way. I would actually prefer simple, not condensed code, so I have a chance to understand it and adjust it if neccessary.
Best
MartinG
=====================
//----------------------------------------------PLOT DATA-----------------------------
Variable/G Scan_start_temp, Scan_end_temp
Variable/G Smth_factor // This is a factor that can be set in the panel to control the level of smoothing of raw data before differentiating
Variable/G Th, Tmax, Tfr, Tmax_signal, Tmin, Tmin_signal, Th_slope,Tfr_slope
Wave XWave
Duplicate/O XWave Scan_temp // Make new wave with temp instead of "cycles"
Scan_temp = XWave + Scan_start_temp -1 // Make new wave with temp instead of "cycles"
Duplicate/O A1,A1_smth;DelayUpdate //Smooth Wave in copy "Wave"_smth applying selected "Smth_factor"
Smooth Smth_factor, A1_smth;DelayUpdate //Smooth Wave in copy "Wave"_smth applying selected "Smth_factor"
Differentiate A1_smth/X=Scan_temp/D=A1_smth_DIF;DelayUpdate // Differentiate smoothed curve
A1_smth_DIF = -1*A1_smth_DIF // Differentiate smoothed curve
Display /W=(200.25,42.5,522.75,293) A1 vs Scan_temp //as "TSA"
AppendToGraph A1_smth vs Scan_temp
AppendToGraph/L=dif A1_smth_DIF vs Scan_temp
ModifyGraph lStyle(A1_smth_DIF)=3
ModifyGraph rgb(A1_smth)=(0,0,52224),rgb(A1_smth_DIF)=(0,52224,0)
ModifyGraph zero(bottom)=1,zero(dif)=1
ModifyGraph mirror=2
ModifyGraph lblPos(left)=57,lblPos(bottom)=37,lblPos(dif)=57
ModifyGraph lblLatPos(dif)=-1
ModifyGraph freePos(dif)={0,bottom}
ModifyGraph axisEnab(left)={0,0.68}
ModifyGraph axisEnab(dif)={0.7,1}
Label left "Fluorescence signal (RFU)"
Label bottom "Temp / DegC"
Label dif "dRFU/dT "
TextBox/C/N=text0/A=MC/X=-27.60/Y=-16.10 "\\Z08\r\\s(A1) Raw data\r\\s(A1_smth) Smoothed data\r\\s(A1_smth_DIF) derivative"
Cursor/P A A1_smth (Th-Scan_start_temp) ;Cursor/P B A1_smth (Tfr-Scan_start_temp); doUpdate
ShowInfo
//---------Calculate Tmax, Tmin, Th, .....
WaveStats/Q A1_smth
Tmax = V_maxloc + Scan_start_temp - 1
Tmax_signal = V_max
Tmin = V_minloc + Scan_start_temp - 1
Tmin_signal = V_min
WaveStats/Q A1_smth_DIF
Th = V_minloc + Scan_start_temp - 1
Th_slope = V_max
Tfr = V_maxloc + Scan_start_temp - 1
Tfr_slope = V_min
//-------------CALCULATE AREA UNDER CURVE-----------------------------
Variable/G AreaUC_ThtoTfr_A1, AreaUC_ThtoTmax_A1, AreaUC_TmintoTfr_A1, AreaUC_TmintoTh_A1, AreaUC_TmintoTmax_A1, AreaUC_TmaxtoTfr_A1
AreaUC_TmintoTh_A1 = areaXY(Scan_temp, A1, Tmin,Th)/1000
AreaUC_TmintoTmax_A1 = areaXY(Scan_temp, A1, Tmin,Tmax)/1000
AreaUC_TmintoTfr_A1 = areaXY(Scan_temp, A1, Tmin,Tfr)/1000
AreaUC_ThtoTmax_A1 = areaXY(Scan_temp, A1, Th,Tmax)/1000
AreaUC_ThtoTfr_A1 = areaXY(Scan_temp, A1, Th,Tfr)/1000
AreaUC_TmaxtoTfr_A1 = areaXY(Scan_temp, A1, Tmax,Tfr)/1000
End
==================
Function TEST(ctrlName) : ButtonControl
String ctrlName
Variable/G i, j, h
Variable/G Scan_start_temp, Scan_end_temp
Variable/G Smth_factor // This is a factor that can be set in the panel to control the level of smoothing of raw data before differentiating
SVar SWave // "string wave for combined RowColumn
string SWaveR // "string wave" for row A-H
string SWaveC // "string wave" for row 1-12
Wave Wells
Print "START"
j = 0
do
i =1
h = char2num("A") + j // "j" = 65, 66, 67
SwaveR = num2char(h) //convert "j" from e.g. "65" to "A"
do
SWave = SWaveR+num2str(i)
i = i +1
Print SWave
//Duplicate/O $(WaveName(SWave,,)), Temp_smth;DelayUpdate
//Smooth Smth_factor, Temp_smth;DelayUpdate
While(i<=12)
j = j +1
while(j<=7)
Print "END"
End
You can create a wave reference to a wave whose name is stored in a string by
Wave w = $MyWaveName
Then use "w" elsewhere in your code to reference the wave where you were using "A1".
October 7, 2016 at 10:25 am - Permalink
Wonderful, works like a charm and so simple (once you know the solution). Thanks!
October 8, 2016 at 03:21 am - Permalink
Now I have the batch operation and I can execute "WaveStats" on the 92 waves in my measurement. However, I would of course to save these wavestats in new new waves with corresponding names; i.e.
1. For Wave A1 i execute "waveStats" and get the variables printed in the history
2. The procedure then loops to wave A2 and performs wavestats - overwrting the first wavestat variables
How do I easiest save the WaveStats for A1 under new name (e.g. A1_results) so I keep the data when wavestats is run again??
PS. I know I have redundant code (defining waves and variables) inside the loops, but since I am a newbie I try to keep it "self-contained" and then I will (maybe) clean-up later :-)
-----------------------------------------
------------------------------------------
String ctrlName
Variable/G i = 0, row = 65, col = 1
Variable/G Scan_start_temp, Scan_end_temp
Variable/G Smth_factor = 10
Variable/G Th, Tfr, Tmax_signal, Tmin, Tmin_signal, Th_slope,Tfr_slope
Wave XWave
DoWindow/K TSA
KillWaves/A/Z // Clear old stuff
//Display XWave
Duplicate/O XWave Scan_temp // Make new wave with temp instead of "cycles"
Scan_temp = XWave + Scan_start_temp -1 // Make new wave with temp instead of "cycles"
Print "==================="
do
col = 1 // "h" is used for MTP column 1-12. Here it is reset for when loop is re-started with new row (A-H)
do // Outer loop loops Row from A-H (=65 + 1, 2, ...12)
//------------OPERATION ON WAVE SET START------------
String MyWaveNameString
MyWaveNameString = num2char(row)+num2str(col) // Define string containing wave named with variables "i" (A-H) and "j" (1-12)
Wave Temp = $MyWaveNameString // Define temporay wave with name in string "MyWaveName (looping A1 to H12)
String MyWaveName_smth_String
MyWaveName_smth_String = num2char(row)+num2str(col)+"_smth"
Wave TEMP_smth = $MyWaveName_smth_String
Duplicate/O TEMP TEMP_smth
Smooth Smth_factor, TEMP_smth
String MyWaveName_smth_DIF_String
MyWaveName_smth_DIF_String = num2char(row)+num2str(col)+"_smth_DIF"
Wave TEMP_smth_DIF = $MyWaveName_smth_DIF_String
Duplicate/O TEMP_smth TEMP_smth_DIF
Differentiate TEMP_smth/X=Scan_temp/D=TEMP_smth_DIF;DelayUpdate
WaveStats TEMP_smth //PROBLEM: When looping the wavestats are overwritten, and I cannot find a way to save them in a wave called e.g. "num2char(row)+num2str(col)+"_Results"
col = col + 1 // Inner loop increment (column increase from 1 to 2 to 3...)
while(col<=12) // Quit loop when column ("j") reaches 12
row = row +1 // Increase h from 0 to 1 to 2... (A to B to C...)
while(row<=65+7) // Quit loop when h = 8 (row = H)
End
October 9, 2016 at 11:02 pm - Permalink
<igor> code </igor>tags for pasting code snippets.Concerning storing the results of
WaveStatshave a look at its/Wflag.October 9, 2016 at 08:23 am - Permalink
Another option is to use this new string to rename the result from Wavestats with /W flag (Thomas's suggestion).
Wave w = $MyWaveName
// do some stuff
String NewWaveName = MyWaveName + "_stats"
// then do this
WaveStats/W w
Wave/Z M_WaveStats
Rename M_WaveStats $NewWaveName
// or this
Make/O/N=1 $NewWaveName
Wave w1 = $NewWaveName
WaveStats w
w1[0] = V_avg // or whatever you want to store
October 10, 2016 at 12:54 am - Permalink
Thanks for your help - I am slowly progressing towards my goal (and learning a lot on the way :-) ). But I have a new challenge:
I have attached complete procedure and made a comment in the routine below where I am stuck.
Best
Martin
row = 65
do
col = 1 // "h" is used for MTP column 1-12. Here it is reset for when loop is re-started with new row (A-H)
do // Outer loop loops Row from A-H (=65 + 1, 2, ...12)
//------------OPERATION ON WAVE SET: START------------
String WellPosition
WellPosition = num2char(row)+num2str(col) // Define string containing names from A1 to H12 ["i" (A-H) and "j" (1-12)]
Wave Orig = $WellPosition // Define the wave with name in string "WellPosition" i.e. wave called A1, A2, A3...to H12) as "Orig" which is a temporary wave name re-used in each loop
//Copy original wave(s)
Duplicate/O Orig, $num2char(row)+num2str(col)+"_orig" // Make copy of original wave(s) to make sure original wave stays intact
AppendToGraph/W=TSA $num2char(row)+num2str(col)+"_orig" vs Scan_temp // Plot curves in "TSA" window
//Smooth waves and calculate stats (V_min, V_minloc, V_max and V_maxloc)
Duplicate/O Orig, $num2char(row)+num2str(col)+"_smth" // Make copy of orignal wave (with correct number of points) for "smoothed" curve data points
Smooth Smth_factor, $num2char(row)+num2str(col)+"_smth" //"Smooth" wave using factor set in "Panle0"
AppendToGraph/W=TSA_smth $num2char(row)+num2str(col)+"_smth" vs Scan_temp
WaveStats/Q/W $num2char(row)+num2str(col)+"_smth" // Q=Quiet, W=create M_WaveStats wave storing variables
Wave M_WaveStats // Define Wavestats output wave "M_WaveStats" as wave
Duplicate/O M_WaveStats $num2char(row)+num2str(col)+"_smth_stat"
AppendToTable/W=Stats $num2char(row)+num2str(col)+"_smth_stat"
// "CHALLENGE": How do I define V_min, V_minloc, V_max and V_maxloc as variables with name "V_min"+$WellPosition (e.g. V_min_A1)
// so that I can perform AreaXY on the wave for each WellPosition from A1 to H12 between V_minloc and V_maxloc, e.g. something like this: areaXY(Scan_temp, $num2char(row)+num2str(col)+"_smth", V_minloc, V_maxloc)
// ???
//First deriv. wave
Differentiate Orig/X=Scan_temp/D=$num2char(row)+num2str(col)+"_smth_DIF"
AppendToGraph/W=TSA_smth_DIF $num2char(row)+num2str(col)+"_smth_DIF" vs Scan_temp
WaveStats/Q/W $num2char(row)+num2str(col)+"_smth_DIF" // Q=Quiet, W=create M_WaveStats wave storing variables
Wave M_WaveStats //Define M_WaveStats as wave for further referencing
Duplicate/O M_WaveStats $num2char(row)+num2str(col)+"_smth_DIF_stat"
AppendToTable/W=Stats_DIF $num2char(row)+num2str(col)+"_smth_DIF_stat"
//Normalised wave: First deriv. and stats
Duplicate/O $num2char(row)+num2str(col)+"_smth", $num2char(row)+num2str(col)+"_smth_norm" // Make copy of smoothed wave (with correct number of points) for normalised curve data points
Wave NORM = $num2char(row)+num2str(col)+"_smth_norm" // Make temp wave called "NORM" for referencing _smth_norm waves
WaveStats/Q NORM // Find "min" and "max"
NORM = (NORM-V_min)/(V_max-V_min) // Standard normalisation
AppendToGraph/W=TSA_smth_norm NORM vs Scan_temp // Plot normalised. They are plotted as "A1_smth_norm", etc
//DoWindow/K Normal // Show values in table
//Edit/N=Normal // Show values in table
//AppendToTable/W=Normal $num2char(row)+num2str(col)+"_smth_norm" // Show values in table
WaveStats/Q/W $num2char(row)+num2str(col)+"_smth_norm" // Get stats for normalised wave (Might not be used as min is 0 and max is 1 per definition)
Wave M_WaveStats //Define M_WaveStats as wave for further referencing
Duplicate/O M_WaveStats $num2char(row)+num2str(col)+"_smth_norm_stat"
AppendToTable/W=Stats_norm $num2char(row)+num2str(col)+"_smth_norm_stat"
Differentiate $num2char(row)+num2str(col)+"_smth_norm"/X=Scan_temp/D=$num2char(row)+num2str(col)+"_smth_norm_DIF" // Differentiate "NORM" and output as "A1_smth_norm_DIF", etc
AppendToGraph/W=TSA_smth_norm_DIF $num2char(row)+num2str(col)+"_smth_norm_DIF" vs Scan_temp
WaveStats/Q/W $num2char(row)+num2str(col)+"_smth_norm_DIF" // Q=Quiet, W=create M_WaveStats wave storing variables
Wave M_WaveStats //Define M_WaveStats as wave for further referencing
Duplicate/O M_WaveStats $num2char(row)+num2str(col)+"_smth_norm_DIF_stat"
AppendToTable/W=Stats_norm_DIF $num2char(row)+num2str(col)+"_smth_norm_DIF_stat"
//------------OPERATION ON WAVE SET: END------------
col = col + 1 // Inner loop increment (column increase from 1 to 2 to 3...)
while(col<=12) // Quit loop when column ("j") reaches 12
row = row +1 // Increase h from 0 to 1 to 2... (A to B to C...)
while(row<=65+7) // Quit loop when h = 8 (row = H)
//--------------------------------END CALCULATION--------------------------------
End
October 12, 2016 at 12:13 am - Permalink
You don't need to define new variables. You can use V_min, V_minLoc etc. as they are variables stored at the time that you need to call AreaXY. They are also stored in your duplicate of M_WaveStats, you would just need to know the point number. All you need are those two variable plus wave references for the x and y waves.
You can also make a string called newName or something and then do
String newName
newName = WellPosition + "_smth"
Duplicate/O Orig, $newName
Wave w0 = $newName // reference to the wave
Smooth Smth_factor, w0
AppendToGraph/W=TSA_smth w0 vs Scan_temp
WaveStats/Q/W w0
Wave M_WaveStats
// Print, maybe you want to do something else.
Print areaXY(Scan_temp, w0, V_minloc, V_maxloc) // you need to declare Scan_temp above somewhere
newName = WellPosition + "_smth_stat"
Duplicate/O M_WaveStats $newName //... etc
Using newName for the name of duplicated waves will help with readability.
Also, you don't need to keep declaring the M_WaveStats wave, once is OK.
Note: I haven't tested the code suggestion above
October 12, 2016 at 07:55 am - Permalink
Duplicate/O Orig, $num2char(row)+num2str(col)+"_smth" // Make copy of orignal wave (with correct number of points) for "smoothed" curve data points
Smooth Smth_factor, $num2char(row)+num2str(col)+"_smth" //"Smooth" wave using factor set in "Panle0"
AppendToGraph/W=TSA_smth $num2char(row)+num2str(col)+"_smth" vs Scan_temp
WaveStats/Q/W $num2char(row)+num2str(col)+"_smth"
to this:
String smoothWaveName = num2char(row)+num2str(col)+"_smth"
Duplicate/O Orig, $(smoothWaveName) // Make copy of orignal wave (with correct number of points) for "smoothed" curve data points
WAVE smoothWave = $(smoothWaveName)
Smooth Smth_factor, smoothWave //"Smooth" wave using factor set in "Panle0"
AppendToGraph/W=TSA_smth smoothWave vs Scan_temp
WaveStats/Q/W smoothWave
I would make a similar change in many other sections of your code.
There are several reasons why the later is better:
1. In the original code, you are executing the exact same code (
num2char(row)+num2str(col)+"_smth") 4 times. This impacts performance (which may not be noticeable in your case), readability of the code, and makes it more likely that your code has bugs. Say, for example, that you originally typed "_smht" and then copy/pasted that into each place you use it. Then you realized that your code didn't work, and you figured out that you should have written "_smth". So you must make sure that you make that change in 4 places. If you miss one of them, you get a bug.2. In your original code, you need to look up the wave whose name is
num2char(row)+num2str(col)+"_smth"4 times. In most situations there's not much cost to a wave lookup, but if you have thousands of waves in one data folder, wave name lookup can get to be expensive. In my modified code, you are only computing the string once, and then doing one wave name lookup when you initialize the smoothWave wave reference variable(*).3. In my modified code, it's easy to use the debugger to inspect the value of smoothWaveName so you can make sure that you're computing that string correctly. Or you can easily print the value to the history window.
4. In my modified code, if you're using the debugger, you can easily see what the value of smoothWave is and see how many points, etc. it has.
(*) The Duplicate operation actually does a wave name lookup on the destination name as well, but I'm not counting that one.
For more information, execute the following commands on Igor's command line:
DisplayHelpTopic "Accessing Global Variables And Waves"
October 12, 2016 at 08:42 am - Permalink