The inner product between two vectors is an abstract concept used to derive some of the most useful results in linear algebra, as well as nice solutions to several difficult practical problems. It is unfortunately a pretty unintuitive concept, although in certain cases we can interpret it as a measure of the similarity between two vectors.
Table of contents
Before giving a definition of inner product, we need to remember a couple of important facts about vector spaces.
When we use the term "vector" we often refer to an array of numbers, and when we say "vector space" we refer to a set of such arrays. However, if you revise the lecture on vector spaces, you will see that we also gave an abstract axiomatic definition: a vector space is a set equipped with two operations, called vector addition and scalar multiplication, that satisfy a number of axioms; the elements of the vector space are called vectors. In that abstract definition, a vector space has an associated field, which in most cases is the set of real numbers or the set of complex numbers . The elements of the field are the so-called "scalars", which are used in the multiplication of vectors by scalars (e.g., to build linear combinations of vectors).
When we develop the concept of inner product, we will need to specify the field over which the vector space is defined. Moreover, we will always restrict our attention to the two fields and .
We are now ready to provide a definition.
Definition Let be a vector space over . An inner product on is a function that associates to each ordered pair of vectors a complex number, denoted by , which has the following properties.
Positivity:where means that is real (i.e., its complex part is zero) and positive.
Definiteness:
Additivity in first argument:
Homogeneity in first argument:
Conjugate symmetry:where denotes the complex conjugate of .
Although this definition concerns only vector spaces over the complex field , we will use it to develop a theory that applies also to vector spaces defined over the field of real numbers. In fact, when is a vector space over , we just need to replace with in the definition above and pretend that complex conjugation is an operation that leaves the elements of unchanged, so that property 5) becomes
When the inner product between two vectors is equal to zero, that is,then the two vectors are said to be orthogonal.
One of the most important examples of inner product is the dot product between two column vectors having real entries.
Let be the space of all real vectors (on the real field ).
The dot product between two real vectors (which has already been introduced in the lecture on matrix multiplication) iswhere is the transpose of , are the entries of and are the entries of .
Example DefineThen
We need to verify that the dot product thus defined satisfies the five properties of an inner product.
Positivity and definiteness are satisfied because where the equality holds if and only if .
Additivity is satisfied because
The dot product is homogeneous in the first argument because
Finally, (conjugate) symmetry holds because
Another important example of inner product is that between two column vectors having complex entries.
Let be the space of all complex vectors (on the complex field ).
The inner product between two vectors is defined to bewhere is the conjugate transpose of , are the entries of and are the complex conjugates of the entries of .
Example DefineThen
Let us check that the five properties of an inner product are satisfied.
Positivity and definiteness are satisfied because where is the modulus of and the equality holds if and only if .
Additivity is satisfied because
The dot product is homogeneous in the first argument because
Finally, conjugate symmetry holds because
We now present further properties of the inner product that can be derived from its five defining properties introduced above.
We have that the inner product is additive in the second argument:
This is proved as follows:where: in steps and we have used the conjugate symmetry of the inner product; in step we have used the additivity in the first argument.
While the inner product is homogenous in the first argument, it is conjugate homogeneous in the second one:
Here is a demonstration:where: in steps and we have used the conjugate symmetry of the inner product; in step we have used the homogeneity in the first argument.
Below you can find some exercises with explained solutions.
Let be a vector space, and an inner product on . Suppose thatComputeunder the assumption that and are orthogonal.
We can compute the given inner product as follows:where: in step we have used the linearity in the first argument; in step we have used the orthogonality of and , which implies that
Let
Computeusing the inner product of complex arrays defined above.
We have that:
Please cite as:
Taboga, Marco (2021). "Inner product", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/inner-product.
Most of the learning materials found on this website are now available in a traditional textbook format.