linear least squares fit
One of the most common uses of least squares fitting is fitting a straight line to data. Whilst, in general, it is difficult to determine the curve which best fits the data, in this case there is a relatively simple formula which can be used.
Suppose we have a data set . Then the straight line which best fits this set is given as
Being the best fitting line means minimizing the merit function , given as
with respect to the parameters and . Expanding the square, this can be written as
where are as above and
This function is a quadratic polynomial; moreover, from its definition as a sum of squares, it is clear that the highest order terms are positive definite, hence it has a minimum and all that remains is to find that minimum. To do this, we set the derivatives equal to zero to obtain the following equations:
These equations are easily solved to give
substituting in the equation for a straight line, we obtain the answer given above. ∎
Because of the ease with which one can make a least squares fit of a line, this technique is often adapted to fitting other sorts of curves by making a change of variables. Two common cases of this practice are power laws and exponentials.
Suppose that one wants to fit some data to a curve of the form . Making a change of variable and defining , the equation of the curve becomes . One can therefore fit the data set to a straight line.
Suppose that one wants to fit some data to a curve of the form . Making a change of variable , and defining , the equation of the curve becomes . One can therefore fit the data set to a straight line.
Although convenient and common, this procedure can be a cheat because changing variables and making a least squares fit of a line is not the same as making a least squares fit to a curve. The reason for this is that the merit functions are different and will not, in general have a minimum in the same place. However, if the data happen to approximately lie on a power curve or an exponential, then the answer obtained by changing variables and fitting will be an approximation to the correct answer. Depending on what one is doing, this approximation may be good enough or one may use it as a starting point for some algorithm to compute the correct minimum.
|Title||linear least squares fit|
|Date of creation||2013-03-22 17:24:16|
|Last modified on||2013-03-22 17:24:16|
|Last modified by||rspuzio (6075)|