# How error in coefficients are calculated in non-linear curve fitting?

Sat, 04/25/2020 - 12:28 am

After a curve fitting process we get optimised values of coefficients as follows-

**coefficients = value +- SD**

My question is how this SD for coeficients are calculated?

Let us consider I'm fitting my data having** n number of points** with a model containing

**. The wave contaning initial guess of coefficients named as**

*c*no of coefficients*(say). Say the iteration stops after*

**coef***.*

**35**I have assumed the following two cases-

**Case 1.** A matrix * M* contains all the coefficients used for

*as columns (say), i.e., after each iteration a new column gets added to the matrix*

**each iterations***. Thus the dimension of*

**M***will be*

**M***. Now, SD are calculated for each row*

**c x 35***and the results are presented as*

**M(i,:)***where*

**K_i = mean_i +- SD_i***is the i-th coefficient in the*

**K_i***wave.*

**coef**** Case 2. **A matrix

*contains all the coefficients used for*

**M***as columns (say). Thus the dimension of*

**each data points***will be*

**M***. After each iteration the values in the matrix*

**c x n***are replaced. Now, SD are calculated for each row*

**M***and the results are presented as*

**M(i,:)***where*

**K_i = mean_i +- SD_i***is the i-th coefficient in the*

**K_i***wave.*

**coef**Now if the case 1 is true then bag guess also contributes to the optimised coefficient values (in SD). I believe this is not the case. Also, in certain cases no of iteration might be very low and then SD would be meaningless.

But if the case 2 is true, then both the above situations will not arise as at the final iteration we will have a distribution of coefficients most suitable for each data points and hence, their SD is also meaningful. Currently, I'm considering the case 2 as true.

Please, clarify whether I'm thinking in right direction or not.

N.B. According to L-M algorithm, I know a small shift vector is added to the initial guess vector after each iteration (may or may not be for each data points) to lower the * ssq*. So, the above said matrix

*can also keep track of that shift vector, right. But for simplicity, I assumed*

**M***contains coefficient values.*

**M**

Thanks in advance,

Subhamoy

The parameter uncertainties from LM Least Squares are estimated by inverting the Hessian matrix to get the covariance matrix, the leading diagonal of which provides the variance on the parameters. The Hessian matrix contains the second order partial differential of chi2 with respect to all the parameters.

April 25, 2020 at 08:09 pm - Permalink

In reply to The parameter uncertainties… by andyfaff

Do the values in this Hessian matrix get replace after each iteration? What is the dimension of this matrix?

April 27, 2020 at 10:54 am - Permalink

Please read about Levenberg-Marquardt least-squares fitting in Numerical Recipes in C, chapter titled "Modeling of Data" or any other book that covers LM fitting. We find Numerical Recipes to be a very approachable description.

The Hessian is NxM matrix where N is the number of data points and M is the number of fit coefficients that are not held. Each element of the matrix is a partial derivative of the fitting function with respect to a given coefficient, evaluated at the X value corresponding to the row of the matrix.

April 27, 2020 at 01:32 pm - Permalink

In reply to Please read about Levenberg… by johnweeks

Thanks johnweeks...I got my answer....and yes I will certainly go through the reference...thank you andyfaff

April 27, 2020 at 11:03 pm - Permalink

I'm pretty sure the Hessian is square. I thought the Jacobian (J) is evaluated as N*M, with each element holding the partial derivative of the squared difference w.r.t the parameter/data point. The Hessian is then approximated at J^{T}J.

Always happy to be corrected though.

April 28, 2020 at 08:27 pm - Permalink

I stand corrected (literally: I have one of those fancy desks!)

What I described is the Design matrix.

But the intent was still achieved: the Hessian and Design matrices are replaced at every iteration, otherwise there would be no progress in the fit. There are some fancy optimizations (that we don't use unless you use the ODR fit option) where only pieces of the matrices are updated at each iteration.

April 29, 2020 at 10:52 am - Permalink