Averaging part of a 2D wave by index stored in another 1D wave

Dear Support,

I have a logical problem to solve. 

Part of my code is looking for values that fall under certain conditions (namely, I'm looking for elements of a 1D wave that exceeds certain total values and store indexes of that exceedance (puffind.ibw) and the logical values of exceedance (logical.ibw)). Now, I need to average counts_raw.ibw using either logical or puffind indexes in such a manner that the algorithm will take the first batch of columns with the logical==1 and store it in a temporary 2D wave. Then I will take that wave and average each column to get 1 row of values for each column which will be stored in a 2D wave, e.g. averages (then kill the temporary 2D wave). Next,  the algorithm should proceed with the logical to the next bunch of true values and extract another 2D temp wave for averaging. The loop should stop at the end of the logical.ibw or puffind.ibw 

I was trying to get the extraction limits by looking for abrupt changes in puffind.ibw but that didn't work well. In Matlab, I would use variable assignment using logical TRUE/FALSE and cellarray that can store different lengths of variables in one place. What is an alternative in IGOR?

 

I hope you will be able to help me with that. Thank you.

counts_raw.ibw (268.77 KB) logical.ibw (3.04 KB) puffind.ibw (1.24 KB)

I think I understand. I would write a function to do this as follows:

In a loop go through puffind and for each value duplicate the 2D data twice, using conditional statements that copy the data if it exceeds the value (or whatever it is you need) and set the rest as 0, second version will set the values to logical (1 and 0). Use MatrixOP to Sum by column for both and divide the first (sum of columns) by the second (n) to get the average.

There are probably more elegant solutions, but I thought I would chip in.

Hi,

Another option is to use the extract function. 

The Extract operation finds data in srcWave  wherever LogicalExpression  evaluates to TRUE and stores any matching data sequentially in destWave, which will be created if it does not already exist.
Parameters
srcWave  is the name of a wave.
destWave  is the name of a new or existing wave that will contain the result.
LogicalExpression  can use any comparison or logical operator in the expression.
Flags
/FREE    Creates a free destWave (see Free Waves).
    /FREE is allowed only in functions and only if a simple name or wave reference structure field is specified for destWave.
/INDX    Stores the index in destWave  instead of data from srcWave.
/O    Allows destWave to be the same as srcWave  (overwrite source).

 

Andy
 

In reply to by sjr51

sjr51 wrote:

I think I understand. I would write a function to do this as follows:

In a loop go through puffind and for each value duplicate the 2D data twice, using conditional statements that copy the data if it exceeds the value (or whatever it is you need) and set the rest as 0, second version will set the values to logical (1 and 0). Use MatrixOP to Sum by column for both and divide the first (sum of columns) by the second (n) to get the average.

There are probably more elegant solutions, but I thought I would chip in.

 

The problem here is that I need to average each bundle of data (when the logical==1) by the numbers of elements from first 1 to the last 1 of that bundle and then proceed to the next bundle of true values and do the averaging again. I cannot have just one average from the selected data.

I have also tried the extract but it cannot extract each bundle to a separate (or temporary wave) every time the group of true values is found in logical...Also, you cannot index source, dest or logical during the process of extraction.

 

Any other solution? 

 

OK it seems I didn't understand what you're trying to do. How about this: logical is 683 rows, counts_raw is 683 rows by 50 cols, so you want to average the rows where logical == 1?

// repeat logical across columns to make a logical matrix
MatrixOp/O matA = colRepeat(logical,50)
// where there was a 1 you keep the value, where it is 0 it becomes NaN
MatrixOp/O matB = counts_raw / matA
// You can average the rows by averaging columns after transposing the matrix
MatrixOp/O result = averageCols(MatB^t)

 

Hi,

I am also not completely clear on the objective, but I will give it a trial.

I took your counts_raw wave (632x50), logical (632 pnts), and your puffind (221 points with data that goes from 0-632).

My approach

Transpose your counts_raw to a new wave of 50x632 pmts.

Wavestats/pcst to get stats for each of the 632 columns.

Then duplicate the puffind wave to a new wave, results

results[] =M_wavestats[%avg][puffind[p]]/logical[puffind[p]]

This gives the avg for each point in the puffind wave and a NaN where the logical wave value is 0.

to remove the points with Nans

wavetransform zapnans result

Am I close to what you need?

Andy

It seems hard to avoid any explicit loop if you want to average blocks of data in counts_raw, where logical == 1.

Here is my take on my interpretation of the task (you may want to remove the /FREE flags to see what happens in detail):

function foo(data, logic)
    wave data, logic
   
    // make a range wave that indicates the start/stop indices over which counts_raw is averaged
    Duplicate/FREE/O logic w, range
    Redimension/N=(numpnts(logic) +1) w
    // find the beginning and end of blocks of 1's
    Range = abs(w[p] - w[p+1])
    Range = Range[p] == 1 ? p : NaN
    WaveTransform zapNaNs Range
    // note that start "marker" index needs to be raised by 1
    // even row number = start, uneven = stop
    Range[0, ;2] += 1
   
    // Make result wave, how many blocks are there?
    variable n = numpnts(Range)
    Make/D/O/N=(n/2, Dimsize(data, 1)) M_Result
   
    // loop through Range and average raw data over blocks of rows
    variable i, idx = 0
    for(i=0; i<n; i+=2)
        Duplicate/FREE/R=[range[i], range[i+1]][] data, block
        MatrixOP/FREE avg = averageCols(Block)
        M_Result[idx][] = avg[0][q]
        idx +=1
    endfor
end

Hope this is what you want.

In reply to by ChrLie

ChrLie wrote:

It seems hard to avoid any explicit loop if you want to average blocks of data in counts_raw, where logical == 1.

Here is my take on my interpretation of the task (you may want to remove the /FREE flags to see what happens in detail):

function foo(data, logic)
    wave data, logic
   
    // make a range wave that indicates the start/stop indices over which counts_raw is averaged
    Duplicate/FREE/O logic w, range
    Redimension/N=(numpnts(logic) +1) w
    // find the beginning and end of blocks of 1's
    Range = abs(w[p] - w[p+1])
    Range = Range[p] == 1 ? p : NaN
    WaveTransform zapNaNs Range
    // note that start "marker" index needs to be raised by 1
    // even row number = start, uneven = stop
    Range[0, ;2] += 1
   
    // Make result wave, how many blocks are there?
    variable n = numpnts(Range)
    Make/D/O/N=(n/2, Dimsize(data, 1)) M_Result
   
    // loop through Range and average raw data over blocks of rows
    variable i, idx = 0
    for(i=0; i<n; i+=2)
        Duplicate/FREE/R=[range[i], range[i+1]][] data, block
        MatrixOP/FREE avg = averageCols(Block)
        M_Result[idx][] = avg[0][q]
        idx +=1
    endfor
end

Hope this is what you want.

 

Than you very much! That works very well. I have learned about selective indexing from your code :) Much appreciated! 

Ps. Thank you for all the other answers , you've been a great help too!

Forum

Support

Gallery

Igor Pro 9

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More