Short Notes: Eigendecomposition of a Matrix

A vector \(\mathbf{v}_i \in \mathbb{C}^{n}\) is called the \(i\)-th eigenvector of a matrix \(\mathbf{A} \in \mathbb{R}^{n \times n}\), if it satisfies the simple equation

\[\begin{align} \mathbf{A} \mathbf{v}_i = \lambda_i \mathbf{v}_i, \label{eq:eigvalues} \end{align}\]

for a scalar value \(\lambda_i \in \mathbb{C}\), called an eigenvalue. (Assuming the matrix \(\mathbf{A}\) is real-valued, the eigenvalues and eigenvectors might still be complex.) Let us further assume that the \(n\) eigenvectors of matrix \(\mathbf{A}\) are linearly independent.

We can now ‘horizontally’ stack the eigenvectors into a matrix \(\mathbf{Q} \in \mathbb{C}^{n \times n}\):

\[\begin{align} \mathbf{Q} = \big[\mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_n \big]. \label{eq:Q} \end{align}\]

Multiplying \(\mathbf{A}\) with \(\mathbf{Q}\) gives us:

\[\begin{align} \mathbf{A}\mathbf{Q} = \big[\mathbf{A}\mathbf{v}_1, \mathbf{A}\mathbf{v}_2, \ldots, \mathbf{A}\mathbf{v}_n \big]. \label{eq:AQ} \end{align}\]

If we compare Eq. \eqref{eq:AQ} with Eq. \eqref{eq:eigvalues}, we can see that:

\[\begin{align} \mathbf{A}\mathbf{Q} = \big[\lambda_1\mathbf{v}_1, \lambda_2\mathbf{v}_2, \ldots, \lambda_n\mathbf{v}_n \big]. \label{eq:AQ_2} \end{align}\]

If we now define a diagonal matrix carrying the eigenvalues \(\lambda_i\) as

\[\begin{align} \mathbf{\Lambda} = \begin{bmatrix} \lambda_1 & 0 & \ldots & 0\\ 0 & \lambda_2 & \ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots & \lambda_n \end{bmatrix}, \label{eq:Lambda} \end{align}\]

we see that

\[\begin{align} \mathbf{Q}\mathbf{\Lambda} = \big[\lambda_1\mathbf{v}_1, \lambda_2\mathbf{v}_2, \ldots, \lambda_n\mathbf{v}_n \big] \label{eq:AQ_3} \end{align}\]

which is equal to Eq. \eqref{eq:AQ_2}:

\[\begin{align} \mathbf{Q}\mathbf{\Lambda} = \mathbf{A}\mathbf{Q}. \label{eq:AQ_4} \end{align}\]

One final rearrangement – post-multiplying Eq. \eqref{eq:AQ_4} with \(\mathbf{Q}^{-1}\) – and we are done:

\[\begin{align} \mathbf{A} = \mathbf{Q}\mathbf{\Lambda}\mathbf{Q}^{-1}. \label{eq:eigendecomposition} \end{align}\]

Eq. \eqref{eq:eigendecomposition} is also called the eigendecomposition of matrix \(\mathbf{A}\).

Related posts: Eigendecomposition is the key ingredient behind efficient matrix exponentiation — used to compute sparse Fibonacci subsequences and lagged Fibonacci generators at large indices. It also reveals why solutions to certain Pell equations grow as powers of the golden ratio in Almost-Equal Isosceles Triangles.