best approximation in inner product spaces
The study of best approximations in inner product spaces has a very elegant treatment with profound consequences. Most of the theory of Hilbert spaces depends on this study and several approximation problems are better understood using this techniques and results.
For example: least square fitting, linear regression, approximation of functions by polynomials, among many other problems, can be seen as particular cases of the general study of best approximation in inner product spaces.
Some of the above problems are going to be discussed later in this entry.
1 Existence and Uniqueness
Our fundamental result on the existence and uniqueness of best approximations is the following (we postpone its proof to this attached entry (http://planetmath.org/ProofOfExistenceAndUniquenessOfBestApproximations)):
2 Geometric Interpretation
Theorem - Let be an inner product space, a subspace and . The following statements are equivalent:
is the best approximation of in .
Thus, the best approximation of in a subspace is just the orthogonal projection of in .
3 Calculation of Best Approximations
When the is a complete subspace of , the best approximation can be ”calculated” explicitly. Recall that, in this case, becomes an Hilbert space (since it is complete) and therefore it has an orthonormal basis.
Again, we postpone the proof of the next result to an attached entry.
Theorem - Let be an inner product space and a complete subspace. Let be an orthonormal basis for . Then for every the best approximation of in is given by
One can also write the best approximation in of any other basis (not necessarily an orthonormal one). For simplicity we present here how that can be done when is a finite dimensional subspace of .
Theorem - Let be an inner product space and a finite dimensional subspace. Let be a basis for . Then for every the best approximation of in is given by
where the coefficients are the solutions of the system of equations
There are several applications of the above results. We explore two of them in the following.
4.0.1 - Approximation of functions by polynomials :
Suppose we want to find a polynomial of degree that approximates in the best possible way a given function . We are in fact trying to find a point in the subspace of polynomials of degree that is closest to , i.e. we are trying to find the best approximation of in that subspace.
For example, let . Consider the basis of the subspace of polynomials of degree .
The best approximation of by these polynomials is the function , where the coefficients are the solutions of the system
Instead of polynomials we could approximate by any other of functions using the same procedure.
4.0.2 - Best Fitting Lines :
Suppose we want to find the line that best fits some given points , i.e. the affine function that minimizes .
With this setting we are then looking for the best approximation of the function on the subspace of affine functions.
A base for the subspace of affine functions is given by the functions and .
The best approximation of on this space is the function , where the coefficients are the solutions of the system
Thus, the function obtained by the above procedure provides the line that best fits the data .
|Title||best approximation in inner product spaces|
|Date of creation||2013-03-22 17:32:16|
|Last modified on||2013-03-22 17:32:16|
|Last modified by||asteroid (17536)|
|Defines||approximation by polynomials|
|Defines||best fitting lines|