The four fundamental subspaces

The four fundamental subspaces of a matrix are the ranges and kernels of the linear maps defined by the matrix and its transpose. They are linked to each other by several interesting relations.

Table of contents

Notation
Column space
Null space
Row space
Left-hand null space
How the dimensions of the four subspaces are related
Orthogonal complementarity
Solved exercises
1. Exercise 1

Notation

In what follows, we denote by:

the space of all column vectors;
the space of all column vectors.

For the time being, we are not going to specify whether the entries of the vectors are real or complex.

Furthermore, we consider a matrix , which defines a linear map that associates to any vector the product :

Example Define the matrix [eq4] The associated linear map transforms vectors into vectors. For instance, [eq5] and [eq6]

Column space

The first fundamental subspace is the range of the linear map defined by :which is called column space of .

Example Consider the matrix introduced in the previous example. We can write a product as [eq8] Therefore, the column space of is the span of two column vectors: [eq9]

More in general, the column space of is the span of its columns.

Null space

The second fundamental space is the kernel of the linear map defined by :which is called null space of .

Example We keep defining as in the previous example. There, we have shown that [eq11] from which it is clear that if and only if $s_{1}=s_{3}=0$ , in which case $s_{2}$ can take any value. Thus, all vectors of the null space have the form [eq12] where the scalar can be chosen arbitrarily. In other words, the null space of is the linear span of one vector: [eq13]

Row space

Let us consider a second linear map , defined by

The third fundamental subspace is the range of the linear map defined by $A^{ op }$ :which is called row space of .

Example We continue with the same matrix as in the previous examples. Thus, [eq16] and [eq17] So, the row space of is the span of two vectors: [eq18]

Left-hand null space

The fourth fundamental space is the kernel of the linear map defined by $A^{ op }$ :which is called left-hand null space of .

Example In the previous example, we have that [eq20] Therefore, $A^{ op }t=0$ if and only if $t_{1}=t_{2}=0$ . Thus, the left-hand null space of contains only the zero vector: [eq21]

How the dimensions of the four subspaces are related

The dimensions of the four subspaces are all related to each other.

A product can be written as a linear combination of the columns of :where the coefficients of the linear combination are the entries of . Therefore, the dimension of is the column rank of .

Similarly, a product $A^{ op }t$ can be seen as a linear combination of the rows of with coefficients . Therefore, the dimension of is the row rank of .

We have previously proved that row and column rank coincide, that is,

By the rank-nullity theorem, we have and

By combining (1), (2) and (3), we can get many interesting relations among the dimensions of the four subspaces.

For example, both and are subspaces of and we have

Similarly, and are subspaces of and we have

Example In the previous examples, is a matrix. Thus we have and . We have shown that [eq38] Since the two vectors spanning are linearly independent, we haveMoreover, we have found that [eq41] which impliesThus,

Orthogonal complementarity

When the matrix is real (i.e., its entries are real numbers), not only the dimensions of the four fundamental subspaces are related to each other, but the four spaces form two couples of orthogonal complements.

Proposition If is real, the two subspaces and are orthogonal complements.

Proof

By definition, the orthogonal complement of iswhere is the standard inner product for spaces of real vectors. But if and only if there exists such that $A^{ op }t=r$ . Thus,Moreover,As a consequence, we haveSince and for every , we havewhich, by the definiteness property of the inner product, implies Hence,

Proposition If is real, the two subspaces and are orthogonal complements.

Proof

By definition, the orthogonal complement of isBut if and only if there exists such that . Thus,Moreover,As a consequence, we haveSince $A^{ op }tin S$ and for every , we havewhich, by the definiteness property of the inner product, implies Hence,

As a consequence, the domain and codomain of the linear map can be written as two direct sums: [eq69]

Example In the previous examples we have [eq70] and [eq71] A generic vector can be written as [eq73] where and are scalars. Any vector can be written as [eq75] The inner product between a vector of and a vector of iswhich shows that couples of vectors from the two subspaces are orthogonal, as expected.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Suppose a matrix is full-rank. Find the dimensions of its four subspaces.

Solution

We infer from the dimension of the matrix thatandSince the matrix is full-rank, we haveMoreover, by the rank-nullity theorem, we have and

How to cite

Please cite as:

Taboga, Marco (2021). "The four fundamental subspaces", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/four-fundamental-subspaces.