Mass fitting 2D waves as a background task

Share icon

Sat, 07/12/2008 - 02:55 am

I have a small problem, maybe somebody here can give me some hints:

I have to fit a gauss (or something reasonably close to one) into 2D images. So far, so good. The 2D images come in from of slices of a 3D matrix. Those matrices come from files that get dropped into a server directory.

Now i have two problems:

a) I want Igor to check in regular intervals if there are new files in the directory. But it has to take into account that there _might_ be more files then can be processed to the next time it will check for files, i.e. the background task should wait for the processing loop to finish.

b) These are tons of peaks. I just tried a quick check can get about 25 gauss2D fits per seconds, which is quite fast, but i would like it to be faster. Is there an easy way to make fitting of slices of a 3D matrix threadsafe? I mean, this might be a 9x9x5000 matrix, and i only need the x and y positions. It also doesnt need to be an exact gauss, approximation would also be enough.

thanks in advance,

markus
Do you really need a Gaussian fit? If you just need to find out where the peak is it would be really simple to work out the centroid of the image, or find out the most intense spot. It would take a lot less time.

In any case I don't think that the background task will fire if the processing loop hasn't finished.

July 12, 2008 at 03:08 am - Permalink

Yes, i really need to do it.
Problem is that this is a rather low resolution, and i need the subpixel position.

The reason for the low resolution is that this is already the 2nd step: The first step finds the maximum positions and cuts out the areas of interest for non-realtime evaluation. Also, the Gauss fit allows to easily verify not only the subpixel position, but also to eliminate outliners (i know a lower limit for the reasonable FWHM of the gauss).

While i was thinking about the problem, i thought about following way to parallize the fitting:

Create a threadsafe function for fitting every slice in a 3dwave, and divide the problem into 4 equal-sized parts. Then copy everything into its own subfolder (so you dont collide with the "coef"-files and fit variables. After that call the threadsave function 4 times for each subfolder.
At the end, get the resultfiles from all subfolders, and combine them.

But how to make sure that all 4 threadsafe functions have finished before starting to access the results?

July 12, 2008 at 01:44 pm - Permalink

I think Threadgroupwait should help you there. It will tell you if all of the threads have finished or not. You can find an example by issuing the command:
•Displayhelptopic "Threadsafe Functions and Multitasking"

July 12, 2008 at 03:41 pm - Permalink

There is a demo experiment showing parallel curve fits in threads. Pull down the File menu and select Example Experiments->Curve Fitting->MultipleFitsInThreads

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

July 16, 2008 at 11:15 am - Permalink

Thank you.
For some reason i missed that one completely...
(ok, the fact that the igor online help system is structured like a labyrinth could be a factor, too :) )

July 17, 2008 at 10:26 am - Permalink

Well, didnt have a lot time recently, but now that data aquisition is nearly finished, focus goes back to evaluation...

I started with the above mentioned function as a template, and man, i was blown away!
To explain:

Even with /Q and /N, a normal fit will still pop up that status window for a millisecond, even when called form a function.

Well, it seems this is absolutely, totally limiting the fitting throughput.
I benchmarked fitting a stack of 9x9x5000 (dont ask, the small resolution is there on purpose...).
It didt clock in at just below 6 minutes, using one core, fitting gauss2D functions.

Now the threaded version did finish after about 1.1 seconds. I had to use a timer, after i recovered from my shock, so fast it was.

The results were bit-identical, so nothing missing there.
I also tried increasing the size, and even at 1024 pixels (32x32), it still was below 30 seconds, and thus an order of magnitude faster than the "normal" version on 9x9 pixels.
So is there any way to make the fitting totally silent even when calling form normal functions/command line? As i dont think spending >90% time in the GUI for a 1000 point fit is sensible...

Otoh, this new speed really makes things easier for me :)

July 30, 2008 at 07:36 am - Permalink

To stop the GUI window coming up create a global variable, V_fitoptions in the data folder you start the fit from, and set bit 2.

i.e.
Variable/g V_fitoptions=4

August 1, 2008 at 04:35 am - Permalink

Andy is right about V_fitOptions=4, but it doesn't have to be a global variable. You can simply make it a local variable within the function that calls CurveFit.

By this time perhaps you have been able to time the non-threaded fit with it. It should give you results that take somewhat less time than N*(threaded time), where N is the number of processors used.

One consequence of using a thread is that all GUI in the curve fitting is suppressed.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

August 8, 2008 at 05:01 pm - Permalink