Interval Gaps and Means

How can I tell it to insert the point NaN when there is a time interval gap greater than 5 seconds and then be able to take a mean for the wave not including the Nan value?


I don't quite understand how to tell it to look for a difference between values and then insert a point there

For the mean I am guessing I would tell it to take a mean for points between NaN and NaN or something like that? I just don't know how to do that, plus how would I combine the means between the NaN points to generate one total mean for the wave?


So here is my thoughts for the inserting NaN point

Since != takes a left hand operand and a right hand operand, both of which are scalar numbers. It returns true (1) if the operands are not equal or false (0) if they are equal

Could I somehow tell != to equal the previous point plus 1 second. So if that is not the case it would return a false (0), and then tell it to turn that false into a NaN with something like this

Wave w = wave0 w = w[p]==0 ? NaN : w[p]w = w[p]==0 ? NaN : w[p]


CCNData1.pxp
astrotool wrote:
How can I tell it to insert the point NaN when there is a time interval gap greater than 60 seconds and then be able to take a mean for the wave not including the Nan value?

I don't quite understand how to tell it to look for a difference between values and then insert a point there

For the mean I am guessing I would tell it to take a mean for points between NaN and NaN or something like that? I just don't know how to do that, plus how would I combine the means between the NaN points to generate one total mean for the wave?


I assume, you are marking time in one wave TIME and wanting to set values in another wave DATA based on TIME steps being larger than a certain CUTOFF. Based on my understanding, I think this (untested) should work ...

   DATA = DATA[p] == (TIME[p+1] - TIME[p])>CUTOFF ? NaN : DATA[p]


When calculating means, NaN is ignored.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
well when I load the data from my instrument the time is actually in a wave called TimeAll, and then I go from there.


Is there a way I can make your code into the values at the point, instead of the point itself, so then I subtract the two point's values and if they are >5 then it inserts a Nan point?
astrotool wrote:
... Is there a way I can make your code into the values at the point, instead of the point itself, so then I subtract the two point's values and if they are >5 then it inserts a Nan point?


Unfortunately, I don't fully understand what you mean to test.

Perhaps you could provide a brief summary of the data you have, a (better) description of what you are trying to do with them, and a sample of code that you wrote and that should do what you want (but does not).

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
Ok, sorry about the confusing way of putting it.

I attached my actual IGOR experiment showing a graph that I have made and the data and procedure I use associated with it.

Goals:
-For each different level of SS I need to remove the first 120 seconds of data

-Remove the line that is created between the two different time intervals for each specific SS

-Remove the first 4 minutes of data from when the cycle starts over again and SS 0.2% (red) goes from very high values back down to its stable low values.



So I had been wanting to put NaN values between when the time intervals were different to get rid of the line that connects the data, and then wanted to figure out how to delete the next 120 points when it finds a larger than 1 second time interval, but any way to accomplish the above goals is what I am looking for.

astrotool wrote:
Ok, sorry about the confusing way of putting it.

I attached my actual IGOR experiment showing a graph that I have made and the data and procedure I use associated with it.

Goals:
-For each different level of SS I need to remove the first 120 seconds of data


Determine the point number that corresponds to 120 seconds. Then, redimension the wave to eliminate data from point 0 to this point. Or just set the data you want to eliminate in the wave to NaN.

astrotool wrote:


-Remove the line that is created between the two different time intervals for each specific SS



Use symbols such as dots.

astrotool wrote:


-Remove the first 4 minutes of data from when the cycle starts over again and SS 0.2% (red) goes from very high values back down to its stable low values.



Determine the point value where this occurs. Set the data prior to this point to NaN or redimension the wave correspondingly.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
So far I have come up with this
Function CleanUp(TimeSS02, CCNSS02)
    wave TimeSS02, CCNss02
   

    Duplicate/R=[] Timess02, TimeSS02Cleaned
    TimeSS02Cleaned = Timess02[p] - Timess02[p-500]
    TimeSS02Cleaned = TimeSS02Cleaned == 500 ? TimeSS02Cleaned[p] : NaN
    CCNss02 = TimeSS02Cleaned == 500 ? CCnss02[p] : NaN
End


and it seems to work pretty good.

astrotool wrote:
So far I have come up with this
Duplicate/R=[] Timess05, TimeSS05Cleaned
TimeSS05Cleaned = Timess05[p] - Timess05[p-250]
TimeSS05Cleaned = TimeSS05Cleaned == 250 ? TimeSS05Cleaned[p] : NaN
CCNss05 = TimeSS05Cleaned == 250 ? CCnss05[p] : NaN




Here are three questions to help you clean up your coding:

1) Why is the /R=[] portion included on the Duplicate command?
2) What happens at the line TimeSS05Cleaned = Timess05[p] - Timess05[p-250] when p = 0 (since Timess05 only exists from p=0 and above)?
3) How can you "fix" the above problem (hint: consider that Value[p] - Value[p-x] = Value[p+x] - Value[p])?

Also, while it may work as written, I believe the wave test is better to follow the format ...

  wave = wave[p] == ...


astrotool wrote:

and when I enter it into the command window it works great, but when I make my procedure as:
Function CleanUp(TimeSS05)
    Variable TimeSS05
        Duplicate/R=[] Timess05, TimeSS05Cleaned
        TimeSS05Cleaned = Timess05[p] - Timess05[p-250]
        TimeSS05Cleaned = TimeSS05Cleaned == 250 ? TimeSS05Cleaned[p] : NaN
        CCNss05 = TimeSS05Cleaned == 250 ? CCnss05[p] : NaN
End


it says that the bracket in Timess05[p] is not a valid name or symbol. How do I fix this?


Again, here are three questions to help you figure this out yourself:

1) What do you want pass to CleanUp(...), a variable value or a wave reference? Look at the differences of the two simple functions ...

Function DoAVariable(xvalue)
   variable xvalue

   print xvalue
   return 0
end

Function DoAWave(xwave)
   wave xwave

   xwave = xwave*2
   return 0
end


2) What are you actually passing to CleanUp(...) in the way you have written it, a variable value or a wave reference?
3) Why do you pass something to CleanUp(...) that you then define specifically in the function ... ie, consider the difference between the two functions below ...

Function FixedInput(xvalue)
    variable xvalue

    xvalue = 3
    print xvalue
    return 0
end

Function OpenInput(xvalue)
   variable xvalue

    print xvalue
    return 0
end


Put these as procedures and then type FixedInput(10) and OpenInput(10) on the command line to see the results.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
jjweimer wrote:
astrotool wrote:
So far I have come up with this
Duplicate/R=[] Timess05, TimeSS05Cleaned
TimeSS05Cleaned = Timess05[p] - Timess05[p-250]
TimeSS05Cleaned = TimeSS05Cleaned == 250 ? TimeSS05Cleaned[p] : NaN
CCNss05 = TimeSS05Cleaned == 250 ? CCnss05[p] : NaN




Here are three questions to help you clean up your coding:

1) Why is the /R=[] portion included on the Duplicate command?
2) What happens at the line TimeSS05Cleaned = Timess05[p] - Timess05[p-250] when p = 0 (since Timess05 only exists from p=0 and above)?
3) How can you "fix" the above problem (hint: consider that Value[p] - Value[p-x] = Value[p+x] - Value[p])?



1) I used /R=[] because the Help Browser said "To include all of a given dimension, use /R=[]", I have only had Igor for 3 weeks now so I am doing my best trying to figure this stuff out and so I put in anything the help says if it seems to work

2) When I run the p - (p-250) line it just enters NaN for all the values until point 250, which is what I need it to do as well as put NaN for the gap times

3)I could probably do the code (p+x) - p, but when I was imagining how I wanted it to go over the points I wanted it to consider the point, and then the points before it not after it. Again, I am new to programming anything so I just did it how I thought of it and it seems to work great now
astrotool wrote:

1) I used /R=[] because the Help Browser said "To include all of a given dimension, use /R=[]", I have only had Igor for 3 weeks now so I am doing my best trying to figure this stuff out and so I put in anything the help says if it seems to work

You're probably just misunderstanding the help on this issue (or possibly it's not very clear). You'd only need to include an empty set of brackets if you also wanted to included a subset of a higher dimension. For example, if you have a 2D wave and wanted to duplicate only a single column of the wave, but want to include all rows, you'd want to do something like this:

Duplicate/R=[][1] oldWave, newWave


There's no need to use a set of empty brackets if you want to duplicate the entire wave. I don't believe it will cause any problems, however.
astrotool wrote:

2) When I run the p - (p-250) line it just enters NaN for all the values until point 250, which is what I need it to do as well as put NaN for the gap times


Consider this as an example of a shorter way to set the points 0 to 250 to NaN ...

   make/n=500 dwave = p^2
   dwave[0,250] = NaN
   display dwave


The subsequent code line sets the gap markers to NaN.

--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH