# canonical correlation

Let $X$ be the $(T,n)$ matrix corresponding to the $n$ signals and $Y$ be a $(T,p)$ matrix corresponding to one set of $p$ signals. Time indexes each row of the matrix ($T$ time samples). Let ${\mathrm{\Sigma}}_{11}$ and ${\mathrm{\Sigma}}_{22}$ be the sample covariance matrices of $X$ and $Y$, respectively, and let ${\mathrm{\Sigma}}_{12}={\mathrm{\Sigma}}_{21}^{\prime}$ be the sample covariance matrix between $X$ and $Y$. For simplicity, we suppose that all signals have zero mean.

Canonical correlation analysis (CCA) finds the linear combinations^{} of the column of $X$ and $Y$ that has the largest correlation^{}; i.e., it finds the weight vectors (loadings) $a$ and $b$ that maximize:

$$\rho =\frac{{a}^{\prime}{\mathrm{\Sigma}}_{12}b}{\sqrt{{a}^{\prime}{\mathrm{\Sigma}}_{11}a}\sqrt{{b}^{\prime}{\mathrm{\Sigma}}_{22}b}}.$$ | (1) |

We follow the derivations of Johnson and we do a change of basis: $c={\mathrm{\Sigma}}_{11}^{1/2}a$ and $d={\mathrm{\Sigma}}_{22}^{1/2}b$.

$$\rho =\frac{{c}^{\prime}{\mathrm{\Sigma}}_{11}^{-1/2}{\mathrm{\Sigma}}_{12}{\mathrm{\Sigma}}_{22}^{-1/2}d}{\sqrt{{c}^{\prime}c}\sqrt{{d}^{\prime}d}}$$ | (2) |

By the Cauchy-Schwartz inequality:

$$\rho \le \frac{\sqrt{{c}^{\prime}{\mathrm{\Sigma}}_{11}^{-1/2}{\mathrm{\Sigma}}_{12}{\mathrm{\Sigma}}_{22}^{-1/2}{\mathrm{\Sigma}}_{22}^{-1/2}{\mathrm{\Sigma}}_{21}{\mathrm{\Sigma}}_{11}^{-1/2}c}\sqrt{{d}^{\prime}d}}{\sqrt{{c}^{\prime}c}\sqrt{{d}^{\prime}d}}=\sqrt{\frac{{c}^{\prime}{\mathrm{\Sigma}}_{11}^{-1/2}{\mathrm{\Sigma}}_{12}{\mathrm{\Sigma}}_{22}^{-1}{\mathrm{\Sigma}}_{21}{\mathrm{\Sigma}}_{11}^{-1/2}c}{{c}^{\prime}c}}.$$ | (3) |

The inequality above is an equality when ${\mathrm{\Sigma}}_{22}^{-1/2}{\mathrm{\Sigma}}_{21}{\mathrm{\Sigma}}_{11}^{-1/2}c$ and $d$ are collinear. The right hand side of the expression above is a Rayleigh quotient and it is maximum when $c$ is the eigenvector^{} corresponding to the largest eingenvalue of ${\mathrm{\Sigma}}_{11}^{-1/2}{\mathrm{\Sigma}}_{12}{\mathrm{\Sigma}}_{22}^{-1}{\mathrm{\Sigma}}_{21}{\mathrm{\Sigma}}_{11}^{-1/2}$ (we obtain the other rows by using the other eigenvalues^{} in decreasing magnitude). This results if the basis of the CCA. We can compute the two canonical variables: ${U}_{1}=Xa$ and ${V}_{1}=Yb$.

We can continue this way to find the subsequent vectors

Title | canonical correlation |
---|---|

Canonical name | CanonicalCorrelation |

Date of creation | 2013-03-22 19:16:11 |

Last modified on | 2013-03-22 19:16:11 |

Owner | tony_bruguier (26297) |

Last modified by | tony_bruguier (26297) |

Numerical id | 4 |

Author | tony_bruguier (26297) |

Entry type | Definition |

Classification | msc 62H20 |