Linear spaces (or vector spaces) are sets that are closed with respect to linear combinations.
In other words, a given set is a linear space if its elements can be multiplied by scalars and added together, and the results of these algebraic operations are elements that still belong to .
Linear spaces are defined in a formal and very general way by enumerating the properties that the two algebraic operations performed on the elements of the spaces (addition and multiplication by scalars) need to satisfy.
In order to gradually build some intuition, we start with a narrower approach, and we limit our attention to sets whose elements are matrices (or column and row vectors).
Furthermore, we do not formally enumerate the properties of addition and multiplication by scalars because these have already been derived in previous lectures (see Matrix addition and Multiplication of a matrix by a scalar).
After this informal presentation, we report a fully general and rigorous definition of vector space.
Definition Let be a set of matrices such that all the matrices in have the same dimension. is a linear space if and only if, for any two matrices and belonging to and any two scalars and , the linear combinationalso belongs to .
In other words, when is a linear space, if you take any two matrices belonging to , you multiply each of them by a scalar, and you add together the products thus obtained, then you have a linear combination, which is also a matrix belonging to .
Example Let be the set of all column vectors whose entries are real numbers. Consider two vectors and belonging to . Denote by and the two entries of , and by and the two entries of . A linear combination of and having two real numbers and as coefficients can be written asBut and are real numbers because products and sums of real numbers are also real numbers. Therefore, the two entries of the vectorare real numbers, which implies that the vector belongs to . Since this is true for any couple of coefficients and , is a linear space.
Before giving a rigorous definition of a vector space, we need to introduce fields, which are the sets of scalars used in the multiplication of vectors by scalars.
Definition Let be a set, together with two binary operations , the addition, denoted by and the multiplication, denoted by . The set is said to be a field if and only if, for any ,, the following properties hold:
Associativity of addition:
Commutativity of addition:
Additive identity: there exists an element of , denoted by , such that
Additive inverse: for each , there exists an element of , denoted by , such that
Associativity of multiplication:
Commutativity of multiplication:
Multiplicative identity: there exists an element of , denoted by , such that
Multiplicative inverse: for each , there exists an element of , denoted by , such that
Distributive property:
As you can see, these are the usual properties satisfied by the addition and multiplication of real numbers, which we studied when we were in school. They are also satisfied by the addition and multiplication of complex numbers.
In other words, both and , equipped with their usual operations, are fields. These are also the only two fields you will encounter in these lectures.
Nonetheless, the abstract definition is useful because it allows us to derive results that are valid for fields in general and that can be applied, when needed, both to and to .
We are now ready to define vector spaces.
Definition Let be a field and let be a set equipped with an operation , called vector addition and denoted by , and another operation , called scalar multiplication and denoted by . The set is said to be a linear space (or vector space) over if and only if, for any and any , the following properties hold:
Associativity of vector addition:
Commutativity of vector addition:
Additive identity: there exists a vector , such that
Additive inverse: for each , there exists an element of , denoted by , such that
Compatibility of multiplications:
Multiplicative identity: if is the multiplicative identity in , then
Distributive property w.r.t. vector addition:
Distributive property w.r.t. field addition:
The elements of a vector space are called vectors and those of its associated field are called scalars.
Note that, in the definition above, when we write and , we mean that the two operations are defined on all of and and always give results in .
Thus, we are implicitly assuming thatwhich is equivalent to the requirement of closure with respect to linear combinations made in our previous informal definition of vector space.
Also note also that we have used the same symbols ( and ) for the operations defined on the field and for those that equip the vector space. Which is which is always clear from the context.
As usual, the symbol can be omitted, both in the context of fields and in that of vector spaces. So, has the same meaning as .
Moreover, the addition sign can be omitted when it is followed by the minus sign of an additive inverse. For example, has the same meaning as .
You can easily verify that any set of matrices (or column or row vectors) equipped with the two operations of matrix addition and multiplication of a matrix by a scalar satisfies all of the above properties, provided that the set is closed with respect to linear combinations.
Example Let be the space of all column vectors having real entries. The addition of two column vectors is defined in the usual manner and any real number can be used to perform the multiplication of vectors by scalars. Stated differently, is the field of scalars. In the lectures on matrix addition and multiplication of a matrix by a scalar we have proved that the various associative, commutative and distributive properties listed above hold. The zero vector that satisfies the additive identity property is a vector whose entries are all equal to zero. By taking a linear combination of two vectors , with scalar coefficients , we obtain another vectorwhose -th entry iswhere and denote the -th entries of and . Because products and sums of real numbers are also real numbers, is a real number. This is true for any . So, is a column vector whose entries are all real numbers. But this means that belongs to . Thus, is closed with respect to linear combinations. Hence it is a linear space.
In other words, the informal and somewhat restrictive definition of vector space that we have provided at the beginning of this lecture is perfectly compatible with the more formal and broader definition given in this section.
Moreover, the first informal definition uses the term "scalars" without specifying the field over which the vector space is defined: the omission is intentional, as the vast majority of results presented in these lectures apply both to linear spaces over the real field and to spaces on .
Example Up to now we have always dealt with real matrices, that is, matrices and vectors whose entries are real numbers. However, everything we have said applies also to complex matrices, that is, matrices whose entries are complex numbers. If we review all the definitions given in previous lectures, we will realize that nowhere we have specified that matrices must have real entries. An important difference is that, in the complex case, multiplication by scalars involves complex scalars, but everything else is a straightforward modification of the real case. For instance, we can take the previous example and replace 1) column vectors having real entries with column vectors having complex entries; 2) the field of scalars with the field . We can leave everything else unchanged and we have a proof of the fact that the space of all column vectors having complex entries is a vector space over .
In the lecture on coordinate vectors, we will also show that the informal definition is much less restrictive than it seems: all the elements of a finite-dimensional vector space can be written as arrays of numbers, so that, in a sense, every finite-dimensional vector space fits the informal definition.
Let's now see an example of vector space that is not directly covered by the more restrictive definition, but is covered by the general definition we have just introduced.
Example A third-order polynomial is a functionwhere the coefficients and the argument are scalars belonging to a field . Consider the space of all third-order polynomials. Let us consider the addition of two polynomials, defined above and defined as follows: The natural way to add them is:Moreover, multiplication of a polynomial by a scalar is performed as follows:It is easy to verify that is a vector space over when it is equipped with the two operations of addition and multiplication by a scalar that we have just defined. Importantly, the additive identity property is satisfied by a polynomial whose coefficients are all equal to zero.
An important concept is that of a linear subspace.
Definition Let be a linear space and a subset of . is a linear subspace of if and only if is itself a linear space, that is, if and only if, for any two vectors and any two scalars and , the linear combinationalso belongs to .
The following is a simple example of a linear subspace.
Example Let be the set of all column vectors whose entries are real numbers. We already know that is a linear space. Let be the subset of composed of all the elements of whose first entry is equal to . Consider two vectors and belonging to the subset . Denote by and the two entries of , and by and the two entries of . By the definition of , we have that and . Therefore, a linear combination of and having two real numbers and as coefficients can be written asThus, the result of this linear combination is a vector whose first entry is equal to and whose second entry is a real number (because products and sums of real numbers are also real numbers). Therefore, the vectoralso belongs to . Since this is true for any couple of coefficients and , is itself a linear space, and hence a linear subspace of .
A perhaps obvious fact is that linear spaces and subspaces are closed with respect to linear combinations of more than two vectors, as illustrated by the following proposition.
Proposition If is a linear (sub)space, then, for any vectors belonging to and any scalars , the linear combinationalso belongs to .
By assumption, closure with respect to linear combinations holds for . We only need to prove that it holds for a generic , given that it holds for . In other words, we need to prove that impliesLet us defineWe have just observed that . Now, we can writeBut is a linear combination of and (both belonging to ) with coefficients and . Therefore, also belongs to , which is what we needed to prove.
Below you can find some exercises with explained solutions.
Let be the set of all column vectors whose entries are real numbers.
Let be the subset of composed of all the elements of whose first entry is twice the second entry.
Show that is a linear subspace of .
From previous examples we know that is a linear space. Now, take any two vectors and belonging to the subset . Denote by and the two entries of , and by and the two entries of . By the definition of , we have that and A linear combination of and with coefficients and can be written asThus, a linear combination of vectors belonging to gives as a result a vector whose second entry is a real number ( is a real number, because products and sums of real numbers are also real numbers) and whose first entry is twice the second entry. Therefore, the vector resulting from the linear combination also belongs to . This is true for any couple of coefficients and . As a consequence, is itself a linear space, and hence a linear subspace of .
Let be a matrix. Let be the set of all vectors that satisfy the equation
Show that is a linear space.
Consider a linear combination of two vectors and belonging to with coefficients and :By the distributive property of matrix multiplication, the product of and this linear combination can be written asBecause and belong to , we have thatAs a consequence,Thus, also the linear combination belongs to , because it satisfies the equation that all vectors of need to satisfy. This is true for any couple of vectors and and for any couple of coefficients and , which implies that is a linear space.
Let be the set of all real column vectors.
Let be the set of all the elements of whose first entry is equal to and whose second entry is equal to .
Verify whether is a linear subspace of .
Consider two vectors and belonging to the subset . Denote by , and the three entries of , and by , and the three entries of . By the definition of , we have that , , and . A linear combination of and with coefficients and can be written asThe second entry of the linear combination () is not necessarily equal to . Therefore, the vectordoes not belong to for some coefficients and . Therefore, is not a linear subspace of .
Please cite as:
Taboga, Marco (2021). "Linear spaces", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/linear-spaces.
Most of the learning materials found on this website are now available in a traditional textbook format.