# Batch curve fitting

sjr51

Sun, 12/16/2012 - 01:54 pm

I would really like some help with batch curve fitting. I have ~200 Y waves with corresponding X waves and would like to fit a user-defined function to each of these between two points. I need to save the coefficients for each wave to use for other things. Obviously, I would like to automate this.

I am using the Batch Curve Fit Procedure from the Help files (6.22A) which I've modified.

My user-defined function has 6 Coefficients. One is constrained and one is held for all fits, two others need to be the Y position at point 5 and 44 (start and end of fit), the other two I have a good guess for and these can float freely. How do I input the two that need to be read from each wave?

Any help would be very much appreciated.

I call the function using:

FitCRtolistofwaves(wavelist("MPD*", ";", ""),wavelist("Time*", ";", ""))

The user-defined function is elsewhere in the procedure window.

Function FitCRToListOfWaves(theYList,theXlist)

String theYList

String theXList

Variable i=0

string aWaveName = ""

string bWaveName = ""

Variable V_fitOptions = 4 // suppress progress window

Variable V_FitError = 0 // prevent abort on error

do

aWaveName = StringFromList(i, theYList)

bWavename = StringFromList(i, theXList)

WAVE/Z aWave = $aWaveName

WAVE/Z bWave = $bWaveName

if (!WaveExists(aWave))

break

endif

// /N suppresses screen updates during fitting

// /Q suppresses history output during fitting

Make/D/N=6/O W_coef

W_coef[0] = {0,1,0.0006,0.0155,600,0.999}

Make/O/T/N=2 T_Constraints

T_Constraints[0] = {"K5 > 0","K5 < 1"}

FuncFit/H="000010"/NTHR=0 /N/Q ChapmanRichards W_coef aWave[5,44] /X=bWave /D/R /C=T_Constraints

WAVE W_coef

// save the coefficients

Duplicate/O W_coef $("cf_"+aWaveName)

// save errors

Duplicate/O W_sigma, $("sig_"+aWaveName)

if (V_FitError != 0)

// Mark the results as being bad

WAVE w = $("cf_"+aWaveName)

w = NaN

WAVE w = $("sig_"+aWaveName)

w = NaN

WAVE w = $("fit_"+aWaveName)

w = NaN

WAVE w = $("Res_"+aWaveName)

w = NaN

V_FitError = 0

endif

i += 1

while(1)

End

String theYList

String theXList

Variable i=0

string aWaveName = ""

string bWaveName = ""

Variable V_fitOptions = 4 // suppress progress window

Variable V_FitError = 0 // prevent abort on error

do

aWaveName = StringFromList(i, theYList)

bWavename = StringFromList(i, theXList)

WAVE/Z aWave = $aWaveName

WAVE/Z bWave = $bWaveName

if (!WaveExists(aWave))

break

endif

// /N suppresses screen updates during fitting

// /Q suppresses history output during fitting

Make/D/N=6/O W_coef

W_coef[0] = {0,1,0.0006,0.0155,600,0.999}

Make/O/T/N=2 T_Constraints

T_Constraints[0] = {"K5 > 0","K5 < 1"}

FuncFit/H="000010"/NTHR=0 /N/Q ChapmanRichards W_coef aWave[5,44] /X=bWave /D/R /C=T_Constraints

WAVE W_coef

// save the coefficients

Duplicate/O W_coef $("cf_"+aWaveName)

// save errors

Duplicate/O W_sigma, $("sig_"+aWaveName)

if (V_FitError != 0)

// Mark the results as being bad

WAVE w = $("cf_"+aWaveName)

w = NaN

WAVE w = $("sig_"+aWaveName)

w = NaN

WAVE w = $("fit_"+aWaveName)

w = NaN

WAVE w = $("Res_"+aWaveName)

w = NaN

V_FitError = 0

endif

i += 1

while(1)

End

Any help would be much appreciated.

I've actually been looking into your problem since this morning. Apologies for letting you think you're alone in the wilderness. I developed a Batch Curve Fit package for Igor 6.30, which is now in beta release, that solves a lot of your problems. You've got some tricky coefficient conditions, however, and it doesn't handle them all. I've been looking into what it would take to solve your problem, but meanwhile you've almost solved it yourself using the function that was the basis of the Batch Curve Fit package! Since you've almost got it yourself I'll first get you to the finish line.

You can set the coefficients you want to have the first and last Y values by setting them in the coefficient wave W_coef and then setting FuncFit's hold flag appropriately. So if coefficients 2 and 3 are to, respectively, take on the first and last Y values that code might look like the following:

String theYList

String theXList

Variable i=0

string aWaveName = ""

string bWaveName = ""

Variable V_fitOptions = 4 // suppress progress window

Variable V_FitError = 0 // prevent abort on error

do

aWaveName = StringFromList(i, theYList)

bWavename = StringFromList(i, theXList)

WAVE/Z aWave = $aWaveName

WAVE/Z bWave = $bWaveName

if (!WaveExists(aWave))

break

endif

// /N suppresses screen updates during fitting

// /Q suppresses history output during fitting

Make/D/N=6/O W_coef

W_coef[0] = {0,1,aWave[5],aWave[44],600,0.999} // Nate set the 2nd and 3rd index coefficient values to the first and last Y values

Make/O/T/N=2 T_Constraints

T_Constraints[0] = {"K5 > 0","K5 < 1"}

FuncFit/H="001110"/NTHR=0 /N/Q ChapmanRichards W_coef aWave[5,44] /X=bWave /D/R /C=T_Constraints // Nate held those coefficients

WAVE W_coef

// save the coefficients

Duplicate/O W_coef $("cf_"+aWaveName)

// save errors

Duplicate/O W_sigma, $("sig_"+aWaveName)

if (V_FitError != 0)

// Mark the results as being bad

WAVE w = $("cf_"+aWaveName)

w = NaN

WAVE w = $("sig_"+aWaveName)

w = NaN

WAVE w = $("fit_"+aWaveName)

w = NaN

WAVE w = $("Res_"+aWaveName)

w = NaN

V_FitError = 0

endif

i += 1

while(1)

End

I changed 2 lines, commented with "// Nate ..."

That should solve your immediate problem. Let me know if it does not. I'll be able to look at it tomorrow morning, US Pacific time.

A cleaner, more generalized way to solve the problem would use an all-at-once fit function and make the first and last Y values a part of the fit equation but not one of the coefficients. You can't do this in a regular fit function because you don't have access to the full Y wave. For help on all-at-once fit functions type the following into your command line:

DisplayHelpTopic "All-At-Once Fitting Functions"

Note that inside the All-At-Once fit function the Y wave parameter has already excluded those parts of the range not included (e.g. 0-4 and 45+ in your case).

Finally, I note that there's a Batch Curve Fit package that I developed that ships with Igor 6.30. It makes a lot of the tasks setting up a batch curve fit easier, and includes a panel which visualizes results. Igor 6.30 is a free upgrade if you're using 6.something. See:

http://www.wavemetrics.net/Updaters/AboutPublicBetaVersions.html

6.30 is also a beta version, for which I'm grateful because I found a bug in the display portion of Batch Curve Fit when using it with an all-at-once fit function. I've fixed the bug but it will not be available until the next beta comes out. Thank you for finding a bug in the beta version without actually using it. Quite a remarkable feat! Also the Batch Curve Fit package does not include constraints, and its not easy to apply your first & last Y condition. I'm going to add constraints immediately, and the first and last Y condition would be doable using an all-at-once fit function.

Good Luck!

Nate

WaveMetrics

Nate Hyde

December 17, 2012 at 04:42 pm - Permalink

Thank you so much for your help! I have just tried your solution and it works really well: only one failed fit out of ~200 waves.

I will give the 6.3 beta release of Igor a try soon, although maybe I should wait for a release with the bug corrected.

Thanks again for your time,

sjr51

December 17, 2012 at 10:56 pm - Permalink

If you're going to do more batch fits I think you'll find the package useful. For one thing it adds a nice panel for viewing results. The bug is fixed on my local version and I'll be done updating the UI to handle constraints in a day or two. If you update to 6.30 and send me an email at support@wavemetrics.com I'll send you the patched up Batch Curve Fit package ahead of the next 6.30 update. This offer goes for anybody else who reads this post. I hesitate to attach it here for fear that a user will grab the attachment BCF procedures at some future date when they are outdated.

Glad I could help!

Nate

Nate Hyde

WaveMetrics

Nate Hyde

December 18, 2012 at 11:08 am - Permalink