Differentiate - irregular spaced data

Just a quick question - I have two 1D waves, with measurements (say distance and time) at irregular times. Does the differentiate function operate on irregular spaced data such as this.  I would use cetral differencing.

While I am not in the position to answer your question with absolute confidence, I would say, yes, you can do that. Just select an x-wave in the Differentiate menu dialogue or use the /X= flag with the Differentiate function. Central differencing is the default here and does not need further options.

In reply to by chozo

Thanks Chozo

I have found from my experiments,  the Differentiate operation doe not work with irregularly spaced points,  unless I am doing something wrong. I used 

Differentiate /METH=0 myVariable /X=myDateTime /D=myDiffferentiatedVariable

  In the plot blue points are wave to be differentiated (left axis) plotted against Date/Time and red is differentiated wave.

 

I see. Could you maybe provide some example experiment file, so that we can test things? I don't immediately see what might be wrong here, but I could imagine that Differentiate may have problems with Date/Time values. Or this is just the way it looks with just a few points to work with...

looks about right to me.

this

Differentiate /METH=0 myVariable /X=myDateTime /D=myDiffferentiatedVariable

is the same as

myDiffferentiatedVariable = (myVariable[p+1] - myVariable[p-1]) / (myDateTime[p+1] - myDateTime[p-1])

It can work only within the precision of the data :)

Edit: looks about right, except that the magnitude and sign of the differentiated data in your plot don't tally with the data. Is there some additional scale factor, like -1000?

In reply to by tony

Tony,

well spotted - see differentiate.pxp which contains code. 

Perhaps the central diffencing is doing what it is supposed to do - I was expecting the differentiated curve to be smoother like the original.  

 

As tony already mentioned, the large jitteriness in the derivative most probably comes from the slight variations in your input data. You could cheat a bit here by smoothing out these variations if this is can be justified. Your data seems to follow a very smooth trend which can be very closely approximated with a polynomial fit. This gives you a very smooth derivative (see the attachment). Of course you would need to think about what errors you are introducing by this approach. But if you just need the trend (slope) of the derivative or something, then this may work.

Differentiate fit.pxp

In reply to by chozo

As you say chozo, a very good fit to the data and smooth diffentiated wave.  This may well be fit  for purpose :-)

I should get a decent value for the slope of the Doppler2 which is better, I think, than smoothing.   These data are one set of many so I need to see if this works in general

CurveFit poly_XOffset 3, mySRangeTot /X=myUTD /D

Out of interest at the command prompt you used 

ReplaceWave /X trace=Doppler2, $""

How does this work? i.e. the $""

Good that this seems to be an option for now. Yes, the fit should work in general. You could also have the fit result as a transient wave (either delete or have a free wave) just for the differentiation if you don't want to clutter up your file.

The ReplaceWave command just got rid of the x-wave in the graph, since Dopper2 has its own (linear) scaling from the fit and does not rely on UTD. The $"" part is an empty assignment for the x-wave which effectively removes the dependence on UTD.

Many people actually struggle with understanding the $ operator. It is basically a way to convert a bunch of characters in to an object reference. You can try to read this to get a grasp on it:

DisplayHelpTopic "String Substitution Using $"

 

In reply to by chozo

Many thanks Chozo and Tony for finding a practical approach to my differention problem. The purpose of the differentiation of my measured data was two-fold; to find the slope of the curve and to see whether it passed through zero.  The fit approach does provide this.  In general the measured data pass slowly through a minimum value and hence provides more points and weighting to the fit.  The V_Chisq  value provides a useful indication of any problems with measured data.