# Positive definite matrix

A square matrix is positive definite if pre-multiplying and post-multiplying it by the same vector always gives a positive number as a result, independently of how we choose the vector.

Positive definite symmetric matrices have the property that all their eigenvalues are positive.

We begin by defining quadratic forms. For the time being, we confine our attention to real matrices and real vectors. At the end of this lecture, we discuss the more general complex case.

Definition Let be a real matrix. A quadratic form in is a transformation where is a vector and is its transpose.

The transformation is a scalar because is a row vector and its product with the column vector gives a scalar as a result.

Example DefineGiven a vector , the quadratic form defined by the matrix is

## Restricting attention to symmetric matrices

When we study quadratic forms, we can confine our attention to symmetric matrices without loss of generality.

Remember that a matrix is symmetric if and only if

Any quadratic form can be written aswhere in step we have used the fact that is a scalar and the transpose of a scalar is equal to the scalar itself.

The matrixis symmetric because

Thus, we have proved that we can always write a quadratic form aswhere is symmetric.

## Definiteness

Square matrices can be classified based on the sign of the quadratic forms that they define.

In what follows iff stands for "if and only if".

Definition Let be the space of all vectors having real entries. A real matrix is said to be:

1. positive definite iff for any non-zero ;

2. positive semi-definite iff for any ;

3. negative definite iff for any non-zero ;

4. negative semi-definite iff for any ;

5. indefinite iff there exist such that and .

Let us make an example.

Example DefineGiven a vector , the quadratic form defined by the matrix isSince the sumwhenever and (hence ), the matrix is positive definite.

## Symmetry in the definition

We note that many textbooks and papers require that a positive definite matrix be symmetric. We keep the requirement distinct: every time that symmetry is needed, we will explicitly say so.

## Focus on positivity

From now on, we will mostly focus on positive definite and semi-definite matrices. The results obtained for these matrices can be promptly adapted to negative definite and semi-definite matrices. As a matter of fact, if is negative (semi-)definite, then is positive (semi-)definite. Thus, results can often be adapted by simply switching a sign.

## A positive definite matrix is full-rank

An important fact follows.

Proposition Let be a matrix. If is positive definite, then it is full-rank.

Proof

The proof is by contradiction. Suppose that is not full-rank. Then its columns are not linearly independent. As a consequence, there is a vector such thatWe can pre-multiply both sides of the equation by and obtainSince is positive definite, this is possible only if , a contradiction. Thus must be full-rank.

## Eigenvalues of a positive definite matrix

The following proposition provides a criterion for definiteness.

Proposition A real symmetric matrix is positive definite if and only if all its eigenvalues are strictly positive real numbers.

Proof

Let us prove the "only if" part, starting from the hypothesis that is positive definite. Let be an eigenvalue of and one of its associated eigenvectors. The symmetry of implies that is real (see the lecture on the properties of eigenvalues and eigenvectors). Moreover, can be chosen to be real since a real solution to the equationis guaranteed to exist (because is rank-deficient by the definition of eigenvalue). Then, we havewhere is the norm of . Since is an eigenvector, . Moreover, by the definiteness property of the norm, . Thus, we havebecause by the hypothesis that is positive definite (we have demonstrated above that the quadratic form involves a real vector , which is required in our definition of positive definiteness). We have proved that any eigenvalue of is strictly positive, as desired. Let us now prove the "if" part, starting from the hypothesis that all the eigenvalues of are strictly positive real numbers. Since is real and symmetric, it can be diagonalized as follows:where is orthogonal and is a diagonal matrix having the eigenvalues of on the main diagonal (as proved in the lecture on normal matrices). The eigenvalues are strictly positive, so we can writewhere is a diagonal matrix such that its -th entry satisfiesfor . Therefore, and, for any vector , we haveThe matrix , being orthogonal, is invertible (hence full-rank). The matrix is diagonal (hence triangular) and its diagonal entries are strictly positive, which implies that is invertible (hence full-rank) by the properties of triangular matrices. The product of two full-rank matrices is full-rank. Therefore, is full-rank. Thus,because . By the positive definiteness of the norm, this implies that and, as a consequence,Thus, is positive definite.

## Eigenvalues of a positive semi-definite matrix

A very similar proposition holds for positive semi-definite matrices.

In what follows positive real number means a real number that is greater than or equal to zero.

Proposition A real symmetric matrix is positive semi-definite if and only if all its eigenvalues are positive real numbers.

Proof

We do not repeat all the details of the proof and we just highlight where the previous proof (for the positive definite case) needs to be changed. The first change is in the "only if" part, where we now havebecause by the hypothesis that is positive semi-definite. The second change is in the "if part", where we have because the entries of are no longer guaranteed to be strictly positive and, as a consequence, is not guaranteed to be full-rank. It follows that

## The complex case

When the matrix and the vectors are allowed to be complex, the quadratic form becomeswhere denotes the conjugate transpose of .

Let be the space of all vectors having complex entries. A complex matrix is said to be:

• positive definite iff is real (i.e., it has zero complex part) and for any non-zero ;

• positive semi-definite iff is real (i.e., it has zero complex part) and for any .

The negative definite and semi-definite cases are defined analogously.

Note that conjugate transposition leaves a real scalar unaffected. As a consequence, if a complex matrix is positive definite (or semi-definite), thenfor any , which implies that . In other words, if a complex matrix is positive definite, then it is Hermitian.

Also in the complex case, a positive definite matrix is full-rank (the proof above remains virtually unchanged).

Moreover, since is Hermitian, it is normal and its eigenvalues are real. We still have that is positive semi-definite (definite) if and only if its eigenvalues are positive (resp. strictly positive) real numbers. The proofs are almost identical to those we have seen for the real case. When adapting those proofs, we just need to remember that in the complex case

## Solved exercises

Below you can find some exercises with explained solutions.

### Exercise 1

Let be a complex matrix and one of its eigenvectors. Can you write the quadratic form in terms of ?

Solution

Let be the eigenvalue associated to . Then,

### Exercise 2

Can you tell whether the matrix is positive definite?

Solution

Let be a vector. Denote its entries by and . Then,Then, if and is positive definite.

### Exercise 3

Suppose that is a complex negative definite matrix. What can you say about the sign of its eigenvalues?

Solution

If is negative definite, thenfor any . As a consequence,In other words, the matrix is positive definite. It follows that the eigenvalues of are strictly positive. If is an eigenvalue of associated to an eigenvector , thenThe latter equation is equivalent toSo, if is an eigenvalue of , then is an eigenvalue of . Thus, the eigenvalues of are strictly negative.