1325 days ago Leave a comment Permalink

Multivariate Normal Conditional Distribution

Previously I made a post about diagonalizing a positive definite symmetric matrix . One of the applications of this within statistics is to diagonalize the variance matrix of a multivariate normal in order to derive conditional distributions. Let

Y\sim N_{p}(m,V)
Consider multiplying Y by the following matrix
\begin{pmatrix} 1 & -V_{12}V_{22}^{-1}\\ 0 & 1\\ \end{pmatrix}Y \sim N_{p}\left( \begin{pmatrix} m_{1} - V_{12}V_{22}^{-1}m_{2} \\ m_{2}\\ \end{pmatrix},\begin{pmatrix} V_{11}-V_{12}V_{22}^{-1}V_{21} & 0 \\ 0 & V_{22}\\ \end{pmatrix}\right)
i.e.
\begin{pmatrix} Y_{1}-V_{12}V_{22}^{-1}Y_{2} \\ Y_{2}\\ \end{pmatrix}\sim N\left( \begin{pmatrix} m_{1} - V_{12}V_{22}^{-1}m_{2} \\ m_{2}\\ \end{pmatrix},\begin{pmatrix} V_{11}-V_{12}V_{22}^{-1}V_{21} & 0 \\ 0 & V_{22}\end{pmatrix} \right)\\
The covariance matrix has now been diagonalized. This is useful because zero covariance implies independence for normally distributed random variables and so it follows that

Y_{1}-V_{12}V_{22}^{-1}Y_{2} \sim N( m_{1} - V_{12}V_{22}^{-1}m_{2} ,V_{11}-V_{12}V_{22}^{-1}V_{21})
Conditiong on Y2 it follows that the conditional distribution is
Y_{1}|Y_{2} \sim N( m_{1} + V_{12}V_{22}^{-1}(Y_{2}-m_{2} ),V_{11}-V_{12}V_{22}^{-1}V_{21})
Notice how the variance of the conditional normal distribution is the marginal variance of Y1 minus something. That is to say the variance of Y1 is reduced given the knowledge of Y2. The variance can be recognized as the Schur complement of the covariance matrix with respect to V22. A similar treatment yields
Y_{2}|Y_{1} \sim N( m_{2} + V_{21}V_{11}^{-1}(Y_{1}-m_{1} ),V_{22}-V_{21}V_{11}^{-1}V_{12})