conditional distribution of multi-variate normal variable
Suppose is defined by for some linear transformation of maximum rank. ( to denotes the transpose operator.)
If , so that is simply a vector in , these formulas reduce to:
If does not have zero mean, then the formula for is modified by adding and replacing by , and the formula for is unchanged.
We split up into two stochastically independent parts, the first part containing exactly the information embodied in . Then the conditional distribution of given is simply the unconditional distribution of the second part that is independent of .
To this end, we first change variables to express everything in terms of a standard multi-variate normal . Let be a “square root” factorization of the covariance matrix , so that:
It is intuitively obvious that orthogonality of the two random normal vectors implies their stochastic independence. To show this formally, observe that the Gaussian density function for factors into a product:
We can construct an orthonormal system of coordinates on under which the components for are completely disjoint from those components of . On the other hand, the densities for , , and remain invariant even after changing coordinates, because they are radially symmetric. Hence the variables and are separable in their joint density and they are independent.
The last term is null because is orthogonal to the range of by definition. (Equivalently, lies in the kernel of .) Thus can always be recovered by a linear transformation on .
Conversely, completely determines , from the analytical expression for that we now give. In general, the orthogonal projection onto the range of an injective transformation is . Applying this to , we have
We see that .
We have proved that conditioning on and are equivalent, and so:
using the defining property of orthogonal projections.
Now we express the result in terms of , and remove the dependence on the transformation (which is not uniquely defined from the covariance matrix):
Of course, the conditional distribution of given is the same as that of , which is multi-variate normal.
|Title||conditional distribution of multi-variate normal variable|
|Date of creation||2013-03-22 18:39:09|
|Last modified on||2013-03-22 18:39:09|
|Last modified by||stevecheng (10074)|