NaN trouble

Share icon

Sun, 09/16/2012 - 12:11 am

I have 16 bit unsigned waves with poissonian noise and particles. I find the particles and make a mask (its 8-bit on creation), and I change the values in this mask from 64 to 1 and and then I would like the particles to be NaN instead of 0. I tried to redimension the mask to 16bit unsigned, and then change all the zeros to NaN, but I get 65535 instead. Far as I can tell, thats a real number and it gets used as a real number in all the operations I have tested so far (I really hope so too, as its a real value that often shows up in my data due to instrument saturation).

What I want is to apply the mask so that the particle are "erased", leaving me with just the (mostly) noise. Then I use wavestats and get the average and variance of the noise. (I can do this now by summing the mask in 0 and 1s state and using this area to calculate the effective area that goes into the average whereas the rest is just zero, but Im not seeing such a quick fix for the variance, hence the usefulness of NaN).

So.. how to get NaN to work in this function. And when does NaN work as a NaN, and when does it work as a number?
I can't follow what you are doing, but here are two ways of converting 0's in an image (matrix) to NaNs:

function test()
    Make/O/N=(6,6) wave0 = mod(p+q, 2 )*enoise(1) // 0's at even points
    MatrixOp  /O  wave0 = Replace(wave0, 0, NaN)
//  ZeroToNaN(wave0)  //  or, try below as an alternative
end

function ZeroToNaN(inwave)
    wave inwave
    inwave =  (inwave[p][q]==0) ? NaN : inwave[p][q]
end


I was a bit surprised to find the MatrixOp method worked; the help file for Replace() was a bit confusing in its referral to maintaining the "number type".

You can't perform any mathematical operations on NaNs. Igor can treat them conveniently for many purposes (e.g. blanking trace portions in a graph, or blanking table entries) but you have to be careful not to ascribe any mathematical properties to them (like testing for equality). User functions detect them by using returning 2 for numtype(NaN).

I believe there was earlier discussion in Exchange about this issue, but it was centered around C-functions for an XOP.

September 16, 2012 at 05:49 am - Permalink

There is no such thing as NaN in integer data. It exists only in floating point, either single- or double-precision. So if you want to store NaNs in your data you must use the Redimension operation to convert it to floating point.

September 16, 2012 at 06:11 am - Permalink

replacing the zeros isnt the problem. its that when I tell it to switch a 0 with NaN, it sticks a 65535 in there instead (as this is apparently the value reserved for NaN in a 16 bit wave, much like 64 = NaN in an 8bit wave) and I am then stuck on how is this number supposed to be a NaN when its a real number as well..

I figured a work-around, but its annoying to copy out all the non-zero values into a seperate wave when you already have them in memory.

EDIT: I see.. thought maybe there was some trick or something, as the /M flag of imageanalyzeparticles automatically stores them in 8-bit and the default is NaN = 64..

September 16, 2012 at 06:14 am - Permalink

daggaz wrote:
replacing the zeros isnt the problem. its that when I tell it to switch a 0 with NaN, it sticks a 65535 in there instead (as this is apparently the value reserved for NaN in a 16 bit wave, much like 64 = NaN in an 8bit wave) and I am then stuck on how is this number supposed to be a NaN when its a real number as well..

I figured a work-around, but its annoying to copy out all the non-zero values into a seperate wave when you already have them in memory.

EDIT: I see.. thought maybe there was some trick or something, as the /M flag of imageanalyzeparticles automatically stores them in 8-bit and the default is NaN = 64..


I am a bit concerned about the notion that "NaN=64". This is indeed mentioned in the ImageAnalyzeParticles documentation in a way that could be misleading. To read the full discussion of this execute:

DisplayHelpTopic "Images and the NaN Concept"


To be clear: there are no number types for which NaN=64. Only in some image applications involving unsigned byte data where NaN can't be represented, we chose the value of 64 to denote transparent pixels. The similarity ends there.


A.G.
WaveMetrics, Inc.

September 17, 2012 at 04:48 pm - Permalink

Thank you for the clarification, Igor.

Edit: Also, there is a typo ;-) "Here is an example of using NaNs to create a WHOLE in an overlay image:"

September 18, 2012 at 12:49 am - Permalink

Igor wrote:

To be clear: there are no number types for which NaN=64. Only in some image applications involving unsigned byte data where NaN can't be represented, we chose the value of 64 to denote transparent pixels. The similarity ends there.


Just to be extra clear.. if I append an 8-bit wave to a table, and enter the value "NaN", it does indeed get changed to 64. NaN = 64 in this case, tho 64 != NaN unless you are going to be using specific hardcoded applications. At least now I know that doing this with integer-valued waves is wrong except in these limited conditions.

September 18, 2012 at 12:57 am - Permalink

daggaz wrote:
At least now I know that doing this with integer-valued waves is wrong except in these limited conditions.


Actually, strictly speaking it's always wrong. The concept of NaN simply does not exist for integers, because every possible bit pattern in an integer maps to a number. In the case of floating point there are special bit patterns reserved for NaN, inf, and -inf.

So even if Igor maps NaN to 64 that is not something to be relied upon because sooner or later there will be data that contains the value '64' and things will break and you will have a hard time figuring out why.

hrodstein's advice is spot on - just promote your waves to double (64 bit floating point) and make use of real NaN's. A double can represent all integers between -2^53 and 2^53 (give or take one) exactly. The only reason when you might be worried about this is if you need to store large amounts of data in memory, but in the vast majority of applications there is no issue with that.

September 18, 2012 at 10:03 am - Permalink

daggaz wrote:

Just to be extra clear.. if I append an 8-bit wave to a table, and enter the value "NaN", it does indeed get changed to 64. NaN = 64 in this case, tho 64 != NaN unless you are going to be using specific hardcoded applications. At least now I know that doing this with integer-valued waves is wrong except in these limited conditions.


This is curious. Which version of IGOR are you running that gives you 64 upon typing NaN for an entry in an unsigned byte wave in a table?

September 18, 2012 at 01:43 pm - Permalink

I did this:
make/B/U junk=p;append junk
displayed it in a table, and typed "nan" in one of the cells. It inserted 255 in Igor 6.30b01.

I believe that is an accident of interpreting the bits in a floating-point NaN as if they were part of an integer. Since it is an accident, you can't count on it. Doing the same with signed byte results in -1. I believe the "magic" bit pattern for a floating-point NaN does contain a bunch of 1's; apparently those 1's are being assigned in this case.

As I said, it's an accident. Don't rely on it.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

September 18, 2012 at 03:32 pm - Permalink

Oh no, i most certainly wouldnt rely on it. In fact, I would do the opposite. If I was going to use one your image functions which relies on this behavior, I would first check the image for pre-existing "NaN" valued integers, and change them by -1, provided that didnt give me additional problems as well..

I am running 6.22a I believe (the latest complete build as of last month or two, not on my work computer right now) on a windows XP machine.

September 19, 2012 at 06:06 am - Permalink