Orthogonal complement

Given a vector space , the orthogonal complement of a subset is the subspace of formed by all the vectors that are orthogonal to the vectors of .

Table of contents

Definition
Orthogonal complements are subspaces
Complementarity
Double complementation
Solved exercises
1. Exercise 1

Definition

Remember that two vectors and are said to be orthogonal if their inner product is equal to zero:

Definition Let be a vector space. Let be a subset of . The orthogonal complement of , denoted by $R^{ot }$ , is

Let us make a simple example.

Example Let be the space of all column vectors having real entries. The inner product between two vectors [eq3] isConsider the set formed by the single vector [eq6] Then, the orthogonal complement of is [eq7] Thus, $R^{ot }$ is formed by all the vectors whose second entry $s_{2}$ is equal to the first entry $s_{1}$ .

Orthogonal complements are subspaces

No matter how the subset is chosen, its orthogonal complement $R^{ot }$ is a subspace, that is, a set closed with respect to taking linear combinations.

Proposition Let be a vector space. Let be a subset of . Then, the orthogonal complement $R^{ot }$ is a subspace of .

Proof

Arbitrarily choose and two scalars $lpha _{1}$ and $lpha _{2}$ . Then, [eq9] where: in step we have used the linearity of the inner product in in its first argument: in step we have used the fact that $s_{1}$ and $s_{2}$ belong to $R^{ot }$ and are therefore orthogonal to each vector of . Thus, we have proved that any linear combination of vectors of $R^{ot }$ is orthogonal to each element of . Hence, it belongs to $R^{ot }$ . As a consequence, $R^{ot }$ is closed with respect to taking linear combinations. Thus, it is a subspace.

Complementarity

Remember that, given two subspaces $S_{1}$ and $S_{2}$ of , their sum is the set

Moreover, when , then we say that the sum is direct and we write $S_{1}oplus S_{2}$ .

When, additionally, we have thatthen the two subspaces $S_{1}$ and $S_{2}$ are said to be complementary subspaces. In other words, two subspaces are complementary if their direct sum gives the whole vector space as a result.

It turns out that if is a subspace, then $R^{ot }$ is one of its complementary subspaces. This is the reason why $R^{ot }$ is called an orthogonal "complement".

Proposition Let be a finite-dimensional vector space. Let be a subspace of . Then, the orthogonal complement $R^{ot }$ is a complementary subspace of .

Proof

We first prove that that is, any vector can be written as a sum of two vectors, one taken from and one taken from $R^{ot }$ . Since and, a fortiori, are finite-dimensional, we can find a basis of . By the Gram-Schmidt process, we can transform it into an orthonormal basis . Moreover, as explained in the lecture on the Gram-Schmidt process, any vector can be decomposed as follows: [eq16] where $arepsilon _{s}$ is orthogonal to all the vectors of the basis . This implies that $arepsilon _{s}$ is orthogonal to every vector of . Therefore, . Moreover, the vector [eq19] belongs to , because , being a subspace, contains all linear combinations of its vectors. In other words, we can write any vector as a sum of a vector of and a vector of $R^{ot }$ . Thus,Now, if a vector belongs to both and $R^{ot }$ , it must be orthogonal to itself, that is,But by the definiteness property of the inner product, the only such vector is the zero vector. Therefore, and the sum is direct. Thus,

In the lecture on complementary subspaces, we have discussed the fact that complements are not necessarily unique, that is, there can be many different complements to a subspace . On the contrary, the orthogonal complement is unique, as $R^{ot }$ is precisely identified by the condition that it must contain all the vectors that satisfy

Double complementation

If we take the orthogonal complement twice, we get back to the original subspace.

Proposition Let be a finite-dimensional vector space. Let be a subspace of . Then,

Proof

By the definition of $R^{ot }$ , any vector is orthogonal to all vectors of $R^{ot }$ and therefore belongs to . Thus,Now, choose any vector . Since is finite-dimensional and is a subspace, and has the decompositionwhere and $sin R^{ot }$ . We have thatbecause , being in , is orthogonal to all elements of $R^{ot }$ . Moreover, because , being in , is orthogonal to all elements of $R^{ot }$ . Therefore, [eq34] By the definiteness property of the inner product, this implies that . Therefore, . Thus, we have proved that the initial assumption that implies that . In other words,By putting the inclusion relations (1) and (2) together, we obtain .

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let be the space of all column vectors having real entries. Let be the subspace containing all vectors of the form [eq38] where , and can be any real numbers satisfying . What is the orthogonal complement of ? In particular, what constraints do the entries of the vectors in $R^{ot }$ need to satisfy? Can you find a vector that spans $R^{ot }$ ?

Solution

The vectors of can be written as [eq40] In other words, is spanned by the two vectors [eq41] The orthogonal complement $R^{ot }$ contains all the vectors that satisfyfor any two scalars and . Since this equation needs to be satisfied for every and , it must be thatDenote by the three components of : [eq45] Then,andThus, the orthogonal complement $R^{ot }$ contains all vectors whose coordinates satisfy the two constraintsandThese constraints are satisfied only by the vectors of the form [eq51] In other words, $R^{ot }$ is spanned by the vector [eq52]

How to cite

Please cite as:

Taboga, Marco (2021). "Orthogonal complement", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/orthogonal-complement.