Search for probability and statistics terms on Statlect
StatLect

Rank of a matrix

by , PhD

The column rank of a matrix is the dimension of the linear space spanned by its columns.

The row rank of a matrix is the dimension of the space spanned by its rows.

Since we can prove that the row rank and the column rank are always equal, we simply speak of the rank of a matrix.

Table of Contents

Column rank

Let us start with a definition.

Definition Let A be a $K	imes L$ matrix. The column rank of A is[eq1]where $A_{ullet l}$ denotes the $l$-th column of A, $QTR{rm}{span}$ denotes the linear span, and $dim $ denotes the dimension.

Remember that the dimension of a linear space is the number of elements of one of its bases, that is, the number of linearly independent vectors that generate the space. So, the column rank of a matrix is the number of linearly independent vectors that generate the same space generated by the columns of the matrix.

Example Consider the matrix [eq2]and the linear space $S$ spanned by its two columns[eq3]that is, the space of all vectors that can be written as linear combinations of $A_{ullet 1}$ and $A_{ullet 2}$. Any vector $sin S$ can be written as[eq4]where $lpha _{1}$ and $lpha _{2}$ are two scalars. Note that the two columns $A_{ullet 1}$ and $A_{ullet 2}$ are linearly dependent because[eq5]Therefore, any vector $sin S$ can be written as a multiple of $A_{ullet 1}$: [eq6]As a consequence, [eq7]is a basis for $S$. It has 1 element. Therefore, the dimension of $S$ and the column rank of A are equal to 1.

Row rank

The definition of row rank is analogous to that of column rank.

Definition Let A be a $K	imes L$ matrix. The row rank of A is[eq8]where $A_{kullet }$ denotes the k-th row of A, $QTR{rm}{span}$ denotes the linear span, and $dim $ denotes the dimension.

In other words, the row rank of a matrix is the dimension of the linear space generated by its rows.

Column rank equals row rank

An important result is that the column rank of a matrix is always equal to its row rank.

Proposition Let A be a $K	imes L$ matrix. Then,[eq9]

Proof

Let [eq10]Then, there exists a basis [eq11]of Kx1 column vectors that spans the same space spanned by the columns of A. Denote by $B$ the $K	imes u$ matrix obtained from the vectors of the basis: [eq12]Each column of A can be expressed as a linear combination of [eq13]. The coefficients of the linear combinations can be collected into a $u	imes L$ matrix $C$ such that[eq14]as we have shown in the lecture on Matrix multiplication and linear combinations. In the same lecture, we have also shown that the rows of the product $BC$ are linear combinations of the rows of $C$, with coefficients taken from $B$. So the span of the rows of A is no larger than the span of the rows of $C$ (because linear combinations of the rows of A can be written as linear combinations of the rows of $C$). There are $u$ rows in $C$. If they are linearly independent, then their span has dimension $u$. Otherwise, it has dimension less than $u$. As a consequence, the row rank of A is less than or equal to $u$ (its column rank). In a completely analogous manner, we prove that the column rank is less than or equal to the row rank: let [eq15]Then, there exists a basis [eq16]of $1	imes L$ row vectors that spans the same space spanned by the rows of A. Denote by $D$ the $v	imes L$ matrix obtained from the vectors of the basis: [eq17]Each row of A can be expressed as a linear combination of [eq18]. The coefficients of the linear combinations can be collected into a $K	imes v$ matrix E such that[eq19]The columns of the product $ED$ are linear combinations of the columns of E, with coefficients taken from $D$. So the span of the columns of A is no larger than the span of the columns of E (because linear combinations of the columns of A can be written as linear combinations of the columns of E). There are $v$ columns in E. If they are linearly independent, then their span has dimension $v$. Otherwise, it has dimension less than $v$. As a consequence, the column rank of A is less than or equal to $v$ (its row rank). Thus, we have proved that[eq20]and[eq21]Therefore, [eq22]

Definition of rank

Having proved that column and row rank coincide, we are now ready to provide the definition of rank.

Definition Let A be a $K	imes L$ matrix. The rank of A, denoted by [eq23], is defined as [eq24]

In other words, the rank of a matrix is the dimension of the linear span of its columns, which coincides with the dimension of the linear span of its rows.

Maximum rank

The following proposition holds.

Proposition Let A be a $K	imes L$ matrix. Then[eq25]

Proof

We have two possible cases. In the first case,[eq26]that is, the number K of rows is less than or equal to the number $L$ of columns. The columns are vectors having K entries. So, the dimension of the space spanned by the columns is less than or equal to K. In other words,[eq27]but [eq28]so that[eq29] In the second case,[eq30]that is, the number $L$ of columns is less than or equal to the number K of rows. The rows are vectors having $L$ entries. So, the dimension of the space spanned by the rows is less than or equal to $L$. In other words,[eq31]but [eq32]so that[eq33]Thus, in both cases,[eq34]

Full-rank

Given the previous results, we can now give a definition of full-rank matrix.

Definition Let A be a $K	imes L$ matrix. Then A is said to be full-rank if and only if[eq35]

Clearly, if A is a square matrix, that is, if $L=K$, then it is full-rank if and only if[eq36]In other words, if A is square and full-rank, then its columns (rows) span the space of all K-dimensional vectors: any K-dimensional vector can be written as a linear combination of the columns (rows) of A.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let A be a $3	imes 2$ matrix. What is the maximum rank that it can have?

Solution

The maximum rank of A is[eq37]

Exercise 2

Define[eq38] What is the rank of A?

Solution

The matrix A has two columns:[eq39]The two columns are linearly independent because neither of them can be written as a scalar multiple of the other. As a matter of fact, they are not multiples. This can be clearly seen from the third entry of $A_{ullet 1}$ which is 0: there is no coefficient that can be multiplied by 0 to obtain 1, the third entry of $A_{ullet 2}$. Therefore, the span of the columns of A has dimension $2$, that is, the column rank of A is equal to $2$, and [eq40]

How to cite

Please cite as:

Taboga, Marco (2021). "Rank of a matrix", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/rank-of-a-matrix.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.