Projection onto the Fixed-Rank Tangent Space

Consider the set of matrices of rank exactly $r$ ,

Copy LaTeX

Let $Y\in\mathcal M_r$ have compact SVD

Copy LaTeX

where $U\in\mathbb R^{m\times r}$ and $V\in\mathbb R^{n\times r}$ have orthonormal columns, and $S\in\mathbb R^{r\times r}$ is invertible. Write

Copy LaTeX

Remark

All orthogonality statements use the Frobenius inner product $\langle X,Z\rangle_F=\operatorname{tr}(X^T Z)$ .

Theorem (Tangent and normal projections)

For every $Z\in\mathbb R^{m\times n}$ , the orthogonal projections onto the tangent and normal spaces at $Y$ are

Copy LaTeX

Block characterization of tangent directions

Choose orthonormal complements $U_\perp$ and $V_\perp$ , and define

Copy LaTeX

In these orthonormal bases, $Y$ has the block form

Copy LaTeX

A matrix $Z$ is tangent at $Y$ if there is a differentiable rank- $r$ curve $Y(t)$ such that

Copy LaTeX

Write the curve and the direction in these bases as

Copy LaTeX

At $t=0$ , $A(0)=S$ and the other three blocks vanish. Since $S$ is invertible, so is $A(t)$ for sufficiently small $t$ . Block elimination gives

Copy LaTeX

Because the curve has rank $r$ , its Schur complement vanishes:

Copy LaTeX

The relation $Y(t)=Y+tZ+o(t)$ gives

Copy LaTeX

Since $B(t),C(t)=O(t)$ and $A(t)^{-1}$ remains bounded, the Schur-complement identity gives $D(t)=O(t^2)$ . Comparing this with $D(t)=tZ_{22}+o(t)$ yields $Z_{22}=0$ .

For the converse, take arbitrary blocks $Z_{11}$ , $Z_{12}$ , and $Z_{21}$ of the appropriate dimensions, and set

Copy LaTeX

Define

Copy LaTeX

Its Schur complement is zero, so $\widetilde Y(t)$ has rank $r$ for small $t$ . Since $D(t)=O(t^2)$ ,

Copy LaTeX

Hence the tangent directions are exactly those with a zero lower-right block.

Tangent and normal blocks

The Frobenius inner product is unchanged by multiplication with the orthogonal matrices $Q_U$ and $Q_V$ , so the four blocks are mutually orthogonal. Since tangent directions have a zero lower-right block, the tangent space is

Copy LaTeX

where $Z_{11}\in\mathbb R^{r\times r}$ , $Z_{12}\in\mathbb R^{r\times(n-r)}$ , and $Z_{21}\in\mathbb R^{(m-r)\times r}$ . Its orthogonal complement is

Copy LaTeX

where $Z_{22}\in\mathbb R^{(m-r)\times(n-r)}$ . Thus the tangent projection keeps the first three blocks, while the normal projection keeps only the lower-right block. For any $Z$ ,

Copy LaTeX

Subtracting the normal component from $Z$ gives

Copy LaTeX

The projectors depend only on the column spaces of $U$ and $V$ , not on $S$ .