StatLect
Index > Matrix algebra

Linear spaces

Linear spaces are sets that are closed with respect to linear combinations. In other words, a given set $S$ is a linear space if its elements can be multiplied by scalars and added together, and the results of these algebraic operations are elements that still belong to $S$.

Table of Contents

Definition

Linear spaces are usually defined in a formal and very general way, without specifying the nature of the elements contained in the linear space, and by enumerating the properties that the two algebraic operations (addition and multiplication by scalars) need to satisfy. In order to keep things simple, we take a narrower approach here, and we limit our attention to sets whose elements are matrices (or vectors, that is, matrices having only one row or one column). Furthermore, we do not formally enumerate the properties of addition and multiplication by scalars, because these have already been derived in previous lectures (see Matrix addition and Multiplication of a matrix by a scalar). Having said that, our definition of a linear space is as follows.

Definition Let $S$ be a set of matrices such that all matrices in $S$ have the same dimension. $S$ is a linear space if and only if, for any two matrices $A_{1}$ and $A_{2}$ belonging to $S$ and any two scalars $lpha _{1}$ and $lpha _{2}$, the linear combination[eq1]also belongs to $S$.

In other words, when $S$ is a linear space, if you take any two matrices belonging to $S$, you multiply each of them by a scalar, and you add together the products thus obtained, then you have a linear combination, which is also a matrix belonging to $S$.

Example Let $S$ be the set of all $2	imes 1$ column vectors whose entries are real numbers (i.e., $S=U{211d} ^{2}$ and the elements of $U{211d} ^{2}$ are written as column vectors). Consider two vectors $A_{1}$ and $A_{2} $ belonging to $S$. Denote by $A_{11}$ and $A_{12}$ the two entries of $A_{1} $, and by $A_{21}$ and $A_{22}$ the two entries of $A_{2}$. A linear combination of $A_{1}$ and $A_{2}$ having two real numbers $lpha _{1}$ and a_2 as coefficients can be written as[eq2]But [eq3] and [eq4] are real numbers, because products and sums of real numbers are also real numbers. Therefore, the two entries of the vector[eq5]are real numbers, which implies that the vector belongs to $S$. Since this is true for any couple of coefficients $lpha _{1}$ and a_2, $S$ is a linear space.

Linear subspace

A related concept is that of a linear subspace.

Definition Let $S$ be a linear space and $S_{1}$ a subset of $S$. $S_{1}$ is a linear subspace of $S$ if and only if $S_{1}$ is itself a linear space, that is, if and only if, for any two matrices $A_{1}$ and $A_{2}$ belonging to $S_{1}$ and any two scalars $lpha _{1}$ and $lpha _{2}$, the linear combination[eq5]also belongs to $S_{1}$.

The following is a simple example of a linear subspace.

Example Let $S$ be the set of all $2	imes 1$ column vectors whose entries are real numbers. From the previous example, we know that $S$ is a linear space. Let $S_{1}$ be the subset of $S$ composed of all the elements of $S$ whose first entry is equal to 0. Consider two vectors $A_{1}$ and $A_{2}$ belonging to the subset $S_{1}$. Denote by $A_{11}$ and $A_{12}$ the two entries of $A_{1} $, and by $A_{21}$ and $A_{22}$ the two entries of $A_{2}$. By the definition of $S_{1}$, we have that $A_{11}=0$ and $A_{21}=0$. Therefore, a linear combination of $A_{1}$ and $A_{2}$ having two real numbers $lpha _{1}$ and a_2 as coefficients can be written as[eq7]Thus, the result of this linear combination is a vector whose first entry is equal to 0 and whose second entry is a real number ([eq4] is a real number, because products and sums of real numbers are also real numbers). Therefore, the vector[eq5]also belongs to $S_{1}$. Since this is true for any couple of coefficients $lpha _{1}$ and a_2, $S_{1}$ is itself a linear space, and hence a linear subspace of $S$.

More details

The following subsections contain more details about linear spaces and subspaces.

More than two matrices in the linear combination

We have defined linear spaces and subspaces as sets that are closed with respect to linear combinations of two matrices. However, it would have been equivalent to require closure with respect to linear combinations of more than two matrices, as illustrated by the following proposition.

Proposition A set of matrices $S$ is a linear (sub)space if and only if, for any n matrices [eq10] belonging to $S$ and any n scalars [eq11], the linear combination[eq12]also belongs to $S$.

Proof

The "if" part is proved by settting $n=2$, which implies that [eq13]The "only if" part can be proved by induction. By assumption, closure holds for $n=2$. We only need to prove that it holds for a generic n, given that it holds for $n-1$. In other words, we need to prove that [eq14]implies[eq15]Let us define[eq16]We have just observed that $Bin S$. Now, we can write[eq17]But $B+lpha _{n}A_{n}$ is a linear combination of $B$ and $A_{n}$ (both belonging to $S$) with coefficients 1 and $lpha _{n}$. Therefore, $B+lpha _{n}A_{n}$ also belongs to $S$, which is what we needed to prove.

Coordinate spaces are linear spaces

An elementary, though extremely important result, is the following.

Proposition The space $U{211d} ^{K}$ of all K-dimensional real vectors is a linear space.

Proof

The proof is a generalization of the proof we have given in the first example above for the space $U{211d} ^{2}$ of all $2$-dimensional real vectors. Without loss of generality, we can assume that the elements of $U{211d} ^{K}$ are Kx1 column vectors. By taking linear combinations of two vectors $A_{1}$ and $A_{2}$ belonging to $U{211d} ^{K}$ with real coefficients $lpha _{1}$ and $lpha _{2}$, we obtain another Kx1 vector[eq5] whose k-th entry is[eq19]where $A_{1k}$ and $A_{2k}$ denote the k-th entries of $A_{1}$ and $A_{2}$. Because products and sums of real numbers are also real numbers, [eq20] is a real number. This is true for any k. So, [eq21] is a Kx1 column vector whose entries are all real numbers. But this means that [eq22] belongs to $U{211d} ^{K}$, which is what we wanted to prove.

Spaces of complex matrices

Up to now, all our examples have dealt with real matrices and vectors, that is, matrices and vectors whose entries are real numbers. However, everything we have said applies also to complex matrices and vectors, that is, matrices and vectors whose entries are complex numbers (review all definitions above, as well as those given in previous lectures, and you will realize that nowhere we have specified that matrices and vectors must have real entries). An important difference is that in the complex case multiplication by scalars involves complex scalars, but everything else is a straightforward modification of the real case. For example, the following result holds.

Proposition The space $U{2102} ^{K}$ of all K-dimensional complex vectors is a linear space.

Proof

Take the proof given above for $U{211d} ^{K}$ and substitute real numbers with complex numbers.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let $S$ be the set of all $2	imes 1$ column vectors whose entries are real numbers. Let $S_{1}$ be the subset of $S$ composed of all the elements of $S$ whose first entry is is twice the second entry. Show that $S_{1}$ is a linear subspace of $S$.

Solution

First of all, we know that $S=U{211d} ^{2}$ is a linear space, because all real coordinate spaces are linear spaces. Now, take any two vectors $A_{1}$ and $A_{2}$ belonging to the subset $S_{1}$. Denote by $A_{11}$ and $A_{12}$ the two entries of $A_{1}$, and by $A_{21}$ and $A_{22}$ the two entries of $A_{2}$. By the definition of $S_{1}$, we have that [eq23] and [eq24]A linear combination of $A_{1}$ and $A_{2}$ with coefficients $lpha _{1}$ and a_2 can be written as[eq25]Thus, a linear combination of vectors belonging to $S_{1}$ gives as a result a vector whose second entry is a real number ([eq4] is a real number, because products and sums of real numbers are also real numbers) and whose first entry is twice the second entry. Therefore, the vector resulting from the linear combination also belongs to $S_{1}$. This is true for any couple of coefficients $lpha _{1}$ and a_2. As a consequence, $S_{1}$ is itself a linear space, and hence a linear subspace of $S$.

Exercise 2

Let A be a $2	imes 2$ matrix. Let $S$ be the set of all $2	imes 1$ vectors x that satisfy the equation[eq27]Show that $S$ is a linear space.

Solution

Consider a linear combination of two vectors $x_{1}$ and $x_{2}$ belonging to $S$ with coefficients $lpha _{1}$ and a_2:[eq28]By the distributive property of matrix multiplication, the product of A and this linear combination can be written as[eq29]Because $x_{1}$ and $x_{2}$ belong to $S$, we have that[eq30]As a consequence,[eq31]Thus, also the linear combination [eq32] belongs to $S$, because it satisfies the equation that all vectors of $S$ need to satisfy. This is true for any couple of vectors $x_{1}$ and $x_{2}$ and for any couple of coefficients $lpha _{1}$ and a_2, which implies that $S$ is a linear space.

Exercise 3

Let $S$ be the set of all $3	imes 1$ real column vectors. Let $S_{1}$ be the set of all the elements of $S$ whose first entry is equal to 0 and whose second entry is equal to 1. Verify whether $S_{1}$ is a linear subspace of $S$.

Solution

Consider two vectors $A_{1}$ and $A_{2}$ belonging to the subset $S_{1}$. Denote by $A_{11}$, $A_{12}$ and $A_{13}$ the three entries of $A_{1}$, and by $A_{21}$, $A_{22}$ and $A_{23}$ the two entries of $A_{2}$. By the definition of $S_{1}$, we have that $A_{11}=0$, $A_{12}=1$, $A_{21}=0$ and $A_{22}=1$. A linear combination of $A_{1}$ and $A_{2}$ with coefficients $lpha _{1}$ and a_2 can be written as[eq33]The second entry of the linear combination ([eq34]) is not necessarily equal to 1. Therefore, the vector[eq5]does not belong to $S_{1}$ for any couple of coefficients $lpha _{1}$ and a_2. Therefore, $S_{1}$ is not a linear subspace of $S$.

The book

Most of the learning materials found on this website are now available in a traditional textbook format.