Resampling data: less points for discrete time steps

Share icon

Fri, 01/16/2015 - 06:21 pm

Hi,
I have (say) 1000 points (wave A) collected from 10:00:00 am up to 10:19:59 am (wave B). About 1 point per second, but not exactly.
How can I "resample" those data considering 20 (integer) minutes (wave D), having a single point per single minute (wave C)?
That is, 20 points, 20 minutes. Wave D (1-20 minutes) is trivial.

Wave C should be a sequence of 20 mean values of about 50 points per time (1000/20) from wave A, I guess.
Analysis > Resample, setting Interpolate/Decimate 20/1000, didn't work for me.

What can I do?
That will depend on the format of your data (x and y waves versus a waveform).

I suggest you attach your data waves or an entire Igor experiment containing your data and be more specific about what you want to do.

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

January 17, 2015 at 11:09 pm - Permalink

JimProuty wrote:
That will depend on the format of your data (x and y waves versus a waveform).

I suggest you attach your data waves or an entire Igor experiment containing your data and be more specific about what you want to do.

--Jim Prouty
Software Engineer, WaveMetrics, Inc.


My entire experiment is pretty huge, so I have to attach just a table with 2 waves: measured data, and time of measure (just as a reference).
I need to plot a new wave of (about) 120 points, from Wave A, with respect to another wave of 120 points (from a modelization). 120 comes from time steps adopted; I can change them in 240 (30 seconds) and so on.

Thank you in advance, Jim.
Greetings
Table9.pxp

January 18, 2015 at 07:51 am - Permalink

I see; you have a time-of-measurement wave (Wave_B) that is usually simply one second after another, except that randomly there is a two-second interval instead.

And you have the measurement result in another wave (Wave_A) at only those times.

In addition, these waves contain about 7200 measurements where you want to represent the average using only, say, 120 measurements (one every 60 seconds).

The solution involves two steps:

1) convert your X/Y data into a waveform with the missing measurements replaced by interpolation.
2) resample (or decimate) the waveform by a factor of 60.

Here's the code I wrote to convert your data from XY to a gap-less waveform:

Macro FillGaps(measuredYWave, timesXWave, suffix)
    String measuredYWave
    String timesXWave
    String suffix= "_nogaps"
    Prompt measuredYWave, "Measurement (Y) wave", popup, WaveList("*",";","")
    Prompt timesXWave, "Time (X) wave", popup, WaveList("*",";","")
    Prompt suffix, "suffix for \"no gaps\" waves"
   
    String noGapsWaveform = FillWaveGaps($measuredYWave, $timesXWave, suffix)
    if( strlen(noGapsWaveform) )
        Edit $noGapsWaveform.id
        Display $noGapsWaveform
    endif
End

Function/S FillWaveGaps(yw, xw, suffix)
    Wave/Z yw, xw
    String suffix
   
    if( !WaveExists(yw) || !WaveExists(xw) )
        Print "both waves must exist!"
        return ""
    endif
    Variable suffixLen= strlen(suffix)
    if( (suffixLen < 2) || (suffixLen > 20) )
        Print "use a suffix between 2 and 20 characters (shorter is better)"
        return ""
    endif
   
    String noGapsWfmName = (NameOfWave(yw)+suffix)[0,30]
   
    // compute smallest increment of time
    Differentiate/METH=1 xw/D=tmp_x
    WaveStats/Q/M=0 tmp_x
    Variable minTimeInc = V_Min
    KillWaves/Z tmp_x
   
    // compute time range
    Variable time0 = xw[0]
    Variable timeLast = xw[numpnts(xw)-1]
    Variable numTimes = (timeLast-time0)/minTimeInc
    numTimes = ceil(numTimes)
   
    // Make waveform from XY
    Make/O/D/N=(numTimes) $noGapsWfmName/WAVE=noGaps
    SetScale/P x, time0, minTimeInc, "dat", noGaps
   
    noGaps = interp(x, xw, yw)  // linear interpolation
   
    return noGapsWfmName
End


The resulting Measured_nogaps waveform is then reduced by a factor of 60 using the Resample dialog:

•Duplicate/O Measured_nogaps,Measured_nogaps_samp
•Resample/DOWN=60/N=59/WINF=None Measured_nogaps_samp


Resulting experiment is attached. I have renamed Wave_A and Wave_B to Measured and TimeOfMeasurement for clarity.

--Jim Prouty
Software Engineer, WaveMetrics, Inc.
BiorTable9.pxp

January 19, 2015 at 01:04 pm - Permalink

Thank you so much, Jim.
Needless to say, your macro works perfectly. I have still some issue in resampling waves with an arbitrary number of points, but I think that decimating the total number of points, and interpolating the number of points I want, should work. The filters too are not totally clear to me, but I can manage that.
Thank you again! Bye.

January 27, 2015 at 03:19 pm - Permalink