Averaging over subsets of a timeseries wave

Share icon

Thu, 05/13/2021 - 03:51 am

Hi, 

This forum has been a tremendous help to me. Thanks a lot for all your advice! 

I am currently working on a 10-day dataset that has 1 second time resolution. What I want to do now is to collapse the data into hourly averages. I'm wondering whether there is a quick wave to do this. My currently inelegant way is to run a for-loop over every 60 index points, calculate the average of that, and put the averages in a new wave. Then create a new 1D timeseries where each row is the mid-point of 60 index points of the original timeseries. This way I can finally plot the hourly-averaged timeseries. 

This is getting the job done but I feel like there has to be a more elegant way to do this. Kindly advise. 

Thanks! 

Peeyush 

Here is my quick idea: You could try to recast your dataset into a 2D matrix with 60x60=3600 columns (giving you 3600 points intervals) using Redimension, like Redimension/N=(numpnts(dataset)/3600,3600) dataset. Note that this may lead to problems when your dataset is not nicely dividable by 3600. Afterwards, you can just average all the columns into one point, e.g., by running MatrixOP/O averageddata= averageCols(dataset). 

May 13, 2021 at 04:42 am - Permalink

You would think it would be easy right? I think the built in "Resample" operation can achieve this. Problems arise when you have data that is on an uneven time base, or has missing values. Then you need to have the averaging be time aware, rather than just point number aware.

A really quick and dirty way is to redimension your data into a matrix X hours by 3600 seconds like this (EDIT: like chozo says ;-)) :

Redimension/N=(numpnts(dataWv_1s)/3600,3600) dataWv_1s ; MatrixTranspose dataWv_1s ; MatrixOP/S dataWv_1hr = sumrows(dataWv_1s)/3600

Else, I also just use various loops depending on what I want.

//___________________________________________________________________________________________________  
// average a wave preserving number of data points
Function/WAVE Average2Hz(Wave dataWv, Variable inHz, variable outHz)
   
    Variable n = numpnts(dataWv),j, k=inHz/outHz
   
    Make/N=(n)/FREE outWv = NaN
   
    do
        WaveStats/Q/R=[j,j+k-1]/M=1 dataWv
        outWv[j+(k/2)]=V_avg
        j+=k
    while (j<(n-k))

    return outWv
End
//___________________________________________________________________________________________________  
// average a wave decimating the number of data points by the ratio of data rates
Function/WAVE AverageDown2Hz(Wave dataWv, Variable inHz, variable outHz)

    Variable n = numpnts(dataWv),i,j, k=inHz/outHz
   
    Make/N=(n/k)/FREE outWv = NaN
   
    do
        WaveStats/Q/R=[j,j+k-1]/M=1 dataWv
        outWv[i]=V_avg
        i+=1 ; j+=k
    while (i<(n/k))
   
    return outWv
End
//___________________________________________________________________________________________________
// function to average a wave down over a specified number of seconds
Function/WAVE avg2secs(Wave dataWv, Wave timeWv, variable secs)

    WaveStats/Q/M=1 timeWv
    Variable rate = round(100 * (V_max - V_min) / V_npnts) / 100
   
    Variable n = numpnts(dataWv),i,j=secs/rate
   
    Make/N=(n)/FREE outWv = NaN
   
    do
        WaveStats/Q/R=[i,i+j-1]/M=1 dataWv
        outWv[i+((j-1)/2)]=V_avg
        i+=j
    while (i<(n-j))

    return outWv
End

 

May 13, 2021 at 04:47 am - Permalink

Hi All, 

This is great! Thank you so much, I went the Redimension way and got it to work. The additional methods are also super helpful, I'm going to try to pick them up as well for future use. 

Thanks a ton, 

Peeyush

May 14, 2021 at 01:29 am - Permalink