Linear spaces are sets that are closed with respect to linear combinations. In other words, a given set is a linear space if its elements can be multiplied by scalars and added together, and the results of these algebraic operations are elements that still belong to .

Linear spaces are usually defined in a formal and very general way, without specifying the nature of the elements contained in the linear space, and by enumerating the properties that the two algebraic operations (addition and multiplication by scalars) need to satisfy. In order to keep things simple, we take a narrower approach here, and we limit our attention to sets whose elements are matrices (or vectors, that is, matrices having only one row or one column). Furthermore, we do not formally enumerate the properties of addition and multiplication by scalars, because these have already been derived in previous lectures (see Matrix addition and Multiplication of a matrix by a scalar). Having said that, our definition of a linear space is as follows.

Definition
Let
be a set of matrices such that all matrices in
have the same dimension.
is a **linear space** if and only if, for any two matrices
and
belonging to
and any two scalars
and
,
the linear
combinationalso
belongs to
.

In other words, when is a linear space, if you take any two matrices belonging to , you multiply each of them by a scalar, and you add together the products thus obtained, then you have a linear combination, which is also a matrix belonging to .

Example Let be the set of all column vectors whose entries are real numbers (i.e., and the elements of are written as column vectors). Consider two vectors and belonging to . Denote by and the two entries of , and by and the two entries of . A linear combination of and having two real numbers and as coefficients can be written asBut and are real numbers, because products and sums of real numbers are also real numbers. Therefore, the two entries of the vectorare real numbers, which implies that the vector belongs to . Since this is true for any couple of coefficients and , is a linear space.

A related concept is that of a linear subspace.

Definition
Let
be a linear space and
a subset of
.
is a **linear subspace** of
if and only if
is itself a linear space, that is, if and only if, for any two matrices
and
belonging to
and any two scalars
and
,
the linear
combinationalso
belongs to
.

The following is a simple example of a linear subspace.

Example Let be the set of all column vectors whose entries are real numbers. From the previous example, we know that is a linear space. Let be the subset of composed of all the elements of whose first entry is equal to . Consider two vectors and belonging to the subset . Denote by and the two entries of , and by and the two entries of . By the definition of , we have that and . Therefore, a linear combination of and having two real numbers and as coefficients can be written asThus, the result of this linear combination is a vector whose first entry is equal to and whose second entry is a real number ( is a real number, because products and sums of real numbers are also real numbers). Therefore, the vectoralso belongs to . Since this is true for any couple of coefficients and , is itself a linear space, and hence a linear subspace of .

The following subsections contain more details about linear spaces and subspaces.

We have defined linear spaces and subspaces as sets that are closed with respect to linear combinations of two matrices. However, it would have been equivalent to require closure with respect to linear combinations of more than two matrices, as illustrated by the following proposition.

Proposition A set of matrices is a linear (sub)space if and only if, for any matrices belonging to and any scalars , the linear combinationalso belongs to .

Proof

The "if" part is proved by settting , which implies that The "only if" part can be proved by induction. By assumption, closure holds for . We only need to prove that it holds for a generic , given that it holds for . In other words, we need to prove that impliesLet us defineWe have just observed that . Now, we can writeBut is a linear combination of and (both belonging to ) with coefficients and . Therefore, also belongs to , which is what we needed to prove.

An elementary, though extremely important result, is the following.

Proposition The space of all -dimensional real vectors is a linear space.

Proof

The proof is a generalization of the proof we have given in the first example above for the space of all -dimensional real vectors. Without loss of generality, we can assume that the elements of are column vectors. By taking linear combinations of two vectors and belonging to with real coefficients and , we obtain another vector whose -th entry iswhere and denote the -th entries of and . Because products and sums of real numbers are also real numbers, is a real number. This is true for any . So, is a column vector whose entries are all real numbers. But this means that belongs to , which is what we wanted to prove.

Up to now, all our examples have dealt with real matrices and vectors, that is, matrices and vectors whose entries are real numbers. However, everything we have said applies also to complex matrices and vectors, that is, matrices and vectors whose entries are complex numbers (review all definitions above, as well as those given in previous lectures, and you will realize that nowhere we have specified that matrices and vectors must have real entries). An important difference is that in the complex case multiplication by scalars involves complex scalars, but everything else is a straightforward modification of the real case. For example, the following result holds.

Proposition The space of all -dimensional complex vectors is a linear space.

Proof

Take the proof given above for and substitute real numbers with complex numbers.

Below you can find some exercises with explained solutions.

Let be the set of all column vectors whose entries are real numbers. Let be the subset of composed of all the elements of whose first entry is is twice the second entry. Show that is a linear subspace of .

Solution

First of all, we know that is a linear space, because all real coordinate spaces are linear spaces. Now, take any two vectors and belonging to the subset . Denote by and the two entries of , and by and the two entries of . By the definition of , we have that and A linear combination of and with coefficients and can be written asThus, a linear combination of vectors belonging to gives as a result a vector whose second entry is a real number ( is a real number, because products and sums of real numbers are also real numbers) and whose first entry is twice the second entry. Therefore, the vector resulting from the linear combination also belongs to . This is true for any couple of coefficients and . As a consequence, is itself a linear space, and hence a linear subspace of .

Let be a matrix. Let be the set of all vectors that satisfy the equationShow that is a linear space.

Solution

Consider a linear combination of two vectors and belonging to with coefficients and :By the distributive property of matrix multiplication, the product of and this linear combination can be written asBecause and belong to , we have thatAs a consequence,Thus, also the linear combination belongs to , because it satisfies the equation that all vectors of need to satisfy. This is true for any couple of vectors and and for any couple of coefficients and , which implies that is a linear space.

Let be the set of all real column vectors. Let be the set of all the elements of whose first entry is equal to and whose second entry is equal to . Verify whether is a linear subspace of .

Solution

Consider two vectors and belonging to the subset . Denote by , and the three entries of , and by , and the two entries of . By the definition of , we have that , , and . A linear combination of and with coefficients and can be written asThe second entry of the linear combination () is not necessarily equal to . Therefore, the vectordoes not belong to for any couple of coefficients and . Therefore, is not a linear subspace of .

The book

Most of the learning materials found on this website are now available in a traditional textbook format.

Featured pages

- Permutations
- Poisson distribution
- Moment generating function
- Statistical inference
- Normal distribution
- Convergence in distribution

Explore

Main sections

- Mathematical tools
- Fundamentals of probability
- Probability distributions
- Asymptotic theory
- Fundamentals of statistics
- Glossary

About

Glossary entries

- IID sequence
- Discrete random variable
- Binomial coefficient
- Factorial
- Alternative hypothesis
- Critical value

Share