Using nested loops to summate data

Share icon

Wed, 12/09/2020 - 10:53 am

Hi, new to the forums but would really appreciate some help!

I run experiments of each cell at a number of different conditions (17 to be precise - starting at 1000ms and working down in 50ms increments to 200ms). Each condition generates a separate delimited text file which I load and analyse using a number of wavestats functions, generating over a dozen waves each. I previously saved each condition as a separate PXP and analysed them separately.

I realise now that actually for each 'cell' I could have a single file with multiple data folders. Running a single analysis macro this way will be x17 times quicker! I have sorted the data from separate files into a single file with 17 data folders. I am trying to run some code which runs through the data folders using a while loop, subtracting 50 each time to increment through the filenames. While in each data folder, I want to use another while loop (maybe not the most efficient way) perform further calculations on the waves. I then want to take all of the individual condition waves and summate them somehow - the best way to do this seems to be to concatenate? Your views on my code below much appreciated. I wondered whether there was some issue with using nested loops where it was failing to loop around the outer loop correctly.

At present the sticking point seems to be the 'inner' loop. I get an error 'name already exists as a variable for A. However it shouldn't already exist Any help greatly appreciated. We're doing research on dangerous arrhythmia and Igor Pro has been a really powerful tool so far, but my coding knowledge still has room for improvement!


 

#pragma rtGlobals=3        // Use modern global access method and strict wave access.

Macro Test_folders()

String Name = "Alt"            //Alt is the first part of the folder name as it must start with a letter
Variable/G root:F = 1000        //Sets the descending folder label

do

if (root:F < 199)
break
endif

String/G root:Folder_name = Name + num2str(F)        //Converts the string and numeric variable into a string...

SetDataFolder $Folder_name                //Which then can be accessed

//Body of code here

    Variable A=0
        Make/N=(numpnts(CL)) DI_full

    
    Edit ThresholdXs
    AppendToTable CL, DI_full
    AppendToTable APD30, APD60, APD90
    AppendToTable DI_full
    AppendToTable dia_sarc, sys_sarc, delta_sarc, sarc_perc_change, AMP
        
    do

        if (A > numpnts(CL)-1)
        break
        endif

        DI_full(A+1) = CL(A+1) - APD90(A+1)
        
        A+=1

        while(A<numpnts(CL)-1)
        

//End of body of code

SetDataFolder root:                    //Back to root folder

root:F -= 50                        //F is reduced by 50 (ms)

while(root:F>199)



//Concatenation of all of the waves from various cycle lengths into summary multidimensional waves

Concatenate {root:Alt1000:ThresholdXS, root:Alt950:ThresholdXS, root:Alt900:ThresholdXS, root:Alt850:ThresholdXS, root:Alt800:ThresholdXS, root:Alt750:ThresholdXS, root:Alt700:ThresholdXS, root:Alt650:ThresholdXS, root:Alt600:ThresholdXS, root:Alt550:ThresholdXS, root:Alt500:ThresholdXS, root:Alt450:ThresholdXS, root:Alt400:ThresholdXS, root:Alt350:ThresholdXS, root:Alt300:ThresholdXS, root:Alt250:ThresholdXS, root:Alt200:ThresholdXS}, root:ThresholdXS
Concatenate {root:Alt1000:CL, root:Alt950:CL, root:Alt900:CL, root:Alt850:CL, root:Alt800:CL, root:Alt750:CL, root:Alt700:CL, root:Alt650:CL, root:Alt600:CL, root:Alt550:CL, root:Alt500:CL, root:Alt450:CL, root:Alt400:CL, root:Alt350:CL, root:Alt300:CL, root:Alt250:CL, root:Alt200:CL}, root:CL
Concatenate {root:Alt1000:APD30, root:Alt950:APD30, root:Alt900:APD30, root:Alt850:APD30, root:Alt800:APD30, root:Alt750:APD30, root:Alt700:APD30, root:Alt650:APD30, root:Alt600:APD30, root:Alt550:APD30, root:Alt500:APD30, root:Alt450:APD30, root:Alt400:APD30, root:Alt350:APD30, root:Alt300:APD30, root:Alt250:APD30, root:Alt200:APD30}, root:APD30
Concatenate {root:Alt1000:APD60, root:Alt950:APD60, root:Alt900:APD60, root:Alt850:APD60, root:Alt800:APD60, root:Alt750:APD60, root:Alt700:APD60, root:Alt650:APD60, root:Alt600:APD60, root:Alt550:APD60, root:Alt500:APD60, root:Alt450:APD60, root:Alt400:APD60, root:Alt350:APD60, root:Alt300:APD60, root:Alt250:APD60, root:Alt200:APD60}, root:APD60
Concatenate {root:Alt1000:APD90, root:Alt950:APD90, root:Alt900:APD90, root:Alt850:APD90, root:Alt800:APD90, root:Alt750:APD90, root:Alt700:APD90, root:Alt650:APD90, root:Alt600:APD90, root:Alt550:APD90, root:Alt500:APD90, root:Alt450:APD90, root:Alt400:APD90, root:Alt350:APD90, root:Alt300:APD90, root:Alt250:APD90, root:Alt200:APD90}, root:APD90
Concatenate {root:Alt1000:dia_sarc, root:Alt950:dia_sarc, root:Alt900:dia_sarc, root:Alt850:dia_sarc, root:Alt800:dia_sarc, root:Alt750:dia_sarc, root:Alt700:dia_sarc, root:Alt650:dia_sarc, root:Alt600:dia_sarc, root:Alt550:dia_sarc, root:Alt500:dia_sarc, root:Alt450:dia_sarc, root:Alt400:dia_sarc, root:Alt350:dia_sarc, root:Alt300:dia_sarc, root:Alt250:dia_sarc, root:Alt200:dia_sarc}, root:dia_sarc
Concatenate {root:Alt1000:sys_sarc, root:Alt950:sys_sarc, root:Alt900:sys_sarc, root:Alt850:sys_sarc, root:Alt800:sys_sarc, root:Alt750:sys_sarc, root:Alt700:sys_sarc, root:Alt650:sys_sarc, root:Alt600:sys_sarc, root:Alt550:sys_sarc, root:Alt500:sys_sarc, root:Alt450:sys_sarc, root:Alt400:sys_sarc, root:Alt350:sys_sarc, root:Alt300:sys_sarc, root:Alt250:sys_sarc, root:Alt200:sys_sarc}, root:sys_sarc
Concatenate {root:Alt1000:delta_sarc, root:Alt950:delta_sarc, root:Alt900:delta_sarc, root:Alt850:delta_sarc, root:Alt800:delta_sarc, root:Alt750:delta_sarc, root:Alt700:delta_sarc, root:Alt650:delta_sarc, root:Alt600:delta_sarc, root:Alt550:delta_sarc, root:Alt500:delta_sarc, root:Alt450:delta_sarc, root:Alt400:delta_sarc, root:Alt350:delta_sarc, root:Alt300:delta_sarc, root:Alt250:delta_sarc, root:Alt200:delta_sarc}, root:delta_sarc
Concatenate {root:Alt1000:sarc_perc_change, root:Alt950:sarc_perc_change, root:Alt900:sarc_perc_change, root:Alt850:sarc_perc_change, root:Alt800:sarc_perc_change, root:Alt750:sarc_perc_change, root:Alt700:sarc_perc_change, root:Alt650:sarc_perc_change, root:Alt600:sarc_perc_change, root:Alt550:sarc_perc_change, root:Alt500:sarc_perc_change, root:Alt450:sarc_perc_change, root:Alt400:sarc_perc_change, root:Alt350:sarc_perc_change, root:Alt300:sarc_perc_change, root:Alt250:sarc_perc_change, root:Alt200:sarc_perc_change}, root:sarc_perc_change
Concatenate {root:Alt1000:AMP, root:Alt950:AMP, root:Alt900:AMP, root:Alt850:AMP, root:Alt800:AMP, root:Alt750:AMP, root:Alt700:AMP, root:Alt650:AMP, root:Alt600:AMP, root:Alt550:AMP, root:Alt500:AMP, root:Alt450:AMP, root:Alt400:AMP, root:Alt350:AMP, root:Alt300:AMP, root:Alt250:AMP, root:Alt200:AMP}, root:AMP

    Edit ThresholdXs
    AppendToTable CL
    AppendToTable APD30, APD60, APD90

    AppendToTable dia_sarc, sys_sarc, delta_sarc, sarc_perc_change, AMP

End

 

 

You do not say what version of Igor Pro you have. Some of this presumes version 8 and may not work well or at all for versions below. A few general comments:

* Use functions rather than macros. Add a MENU option to the top of you procedure file to call the function.

* Use for loops rather than do-while loops when possible. The logic in for-loops is easier to follow, especially when your start, step, and end variables are already known.

* Break the inner-most portion into its own separate function call. Test that function on data in a "dummy" folder. Assure that it works properly. Call that function from within the inner loop.

* Use Data Folder References instead of jumping in and out of data folders.

Otherwise ...

* You are aware that you will end up generating 17 tables when you run this, and 17 more again when you run it again, and so on? Perhaps you should eliminate the EDIT and APPENDTOTABLE lines. Make them a separate function that you call when you want only to open the tables to edit.

* Your calculations use the () brackets rather than the [] brackets. You know that the former references the x-value and the latter references point-location in the array?

December 9, 2020 at 11:58 am - Permalink

EDIT: Uh oh, a couple of minutes too late again. ;)

Welcome to the forum. First and foremost: Please try to wrap your code example into a formatted code snipped if possible, as explained here (this makes code much more readable for everybody):

https://www.wavemetrics.com/forum/general/formatting-code-forum-posts

Indeed, there is a lot to improve in the code, but the forum is the perfect place to get suggestions. I actually don't know why you are getting the error, but it's difficult to find out without having your experiment file. Here are some suggestions anyway:

The first thing I would like to heavily recommend is to write Functions instead of Macros. This has the added benefit of error checking while you are writing the code and not after you run it. Also you can stay away from global variables this way. Another thing is that you may want to stay away from do-while loops for these simple countdowns and use for-loops. Do-while loops have the tendency to run forever if you do a mistake. Then, it seems to me you don't even need the inner loop and just having a simple wave assignment will do the job just fine, like DI_full = CL[p] - APD90[p]. Finally, I would separate the task of creating your results and actually displaying them in a table (by the way, what do you need all these tables for?).

Hope that helps.

December 9, 2020 at 12:02 pm - Permalink

At present the sticking point seems to be the 'inner' loop. I get an error 'name already exists as a variable for A. However it shouldn't already exist

The declaration of variable A is within the do loop. (Variable A=0). You should declare the variable outside of both loops, and then just set the value of A (A = 0) before the inner loop.

December 10, 2020 at 09:14 am - Permalink

The multiple definition of A won't happen if you make it a function. And a function will run much faster (that may not be a problem yet).

December 10, 2020 at 02:52 pm - Permalink

kaikin wrote:

At present the sticking point seems to be the 'inner' loop. I get an error 'name already exists as a variable for A. However it shouldn't already exist

The declaration of variable A is within the do loop. (Variable A=0). You should declare the variable outside of both loops, and then just set the value of A (A = 0) before the inner loop.

 

Thanks, but how would I ensure A resets to 1000 if it's not in the outer loop? Each run of the outer loop needs A to start at 1000. Would I use the 'return' command?

December 11, 2020 at 01:49 am - Permalink

johnweeks wrote:

The multiple definition of A won't happen if you make it a function. And a function will run much faster (that may not be a problem yet).

 

Yes, it certainly sounds as if functions are the way to go. I may spend some time converting my many macros into functions. It also sounds as if my technique of working through waves to perform calculations (Z+1) may be redundant using a simple wave assignment ( wave3 = wave1[p] - wave2[p] for example)...

December 11, 2020 at 01:52 am - Permalink

To go with your macro for now, you can write Variable A once at the beginning, and then just use A = 0 to initialize. But yeah, writing functions it the way to go nowadays. You can ask here if you have problems, but of course you should also read the programming intro chapter in the manual.

December 11, 2020 at 02:45 am - Permalink