Least Squares Proplem

## Primary tabs

# Least Squares Proplem

Submitted by Huntelaar on Wed, 06/25/2008 - 13:54

Forums:

Hello forum!

I'm actually working on my masters thesis and i have to deal with the following problem:

A is a overdetermined equations system with entries a(i,j) which are independent random variables with a given mean and a given variance. b is a vector. I have to find a solution(vector) x to the problem

min{ E [ ||Ax-b||^2]} where || denotes the euklidian norm.

Unfortunately, I have no idea how to solve this problem. It would be great if anyone who had experience with such a problem could give me advice or a hint for a book.

Thank you!

- Forums
- Planetary Bugs
- HS/Secondary
- University/Tertiary
- Graduate/Advanced
- Industry/Practice
- Research Topics
- LaTeX help
- Math Comptetitions
- Math History
- Math Humor
- PlanetMath Comments
- PlanetMath System Updates and News
- PlanetMath help
- PlanetMath.ORG
- Strategic Communications Development
- The Math Pub
- Testing messages (ignore)

- Other useful stuff
- Corrections

## Re: Least Squares Proplem

I am aware of the fact that the a's are random variables.

In the evaluation of E[(Ax,Ax)], you will have a sum of terms involving E[(a(i,j)a(n,m)]. Unless the indices are the same, you can use independence to evaluate the term as the product of the means. When the indices are the same, you have a second moment, which is readily obtained from the mean and variance.

For the terms from E[(Ax,b)], you will have E[a(i,j)] terms - i.e. means.

## Re: Least Squares Proplem

Read the life of C.F. Gauss, by Dunnington, or another good author. As an astronomer, physicist and mathematician, in that order, Gauss developed the theory of least squares, finding the orbit of Ceres, the first verified asteroid, mapped in 1801. Finding Ceres' orbit made Gauss famous world-wide.

During Gauss' teaching years, it is said that least squares was his favorite class. That is, there must of been a large body of theory on the topic, or Gauss would have tired of the topic.

## Re: Least Squares Proplem

Have you tried brute force?

Step 1: ||Ax-b||^2=(Ax,Ax)-2(Ax,b)+(b,b)

Step 2: d/dxi(above expression)=0

You will have n linear equations in n unknowns (xi).

Step 3: Solve!!!

## Re: Least Squares Proplem

Hello..... I don't want articles on using MATLAB. I know how to apply this method using MATLAB. I'm interested in the underlying theory. Any suggestions? A google search yields only MATLAB articles.... Please give me advice...

========================================================

Johnson

http://www.globalinternetmarketing.net