 StatLect

# Matrix algebra

This is a set of lecture notes on matrix algebra. Use these lectures for self-study or as a complement to your textbook.

## The algebra of numeric arrays

Matrices, their characteristics, introduction to some special matrices

Obtained by multiplying matrices by scalars, and by adding them together

How to multiply a matrix by a scalar, definition and properties of scalar multiplication

It plays in matrix multiplication the same role played by 1 in the multiplication of numbers

How to multiply two matrices, definition and properties of multiplication

Addition, scalar multiplication and multiplication of block matrices can be performed on their blocks

A matrix that has been partitioned into smaller submatrices

## Linear spaces

This lecture introduces one of the central concepts in matrix algebra

Sets of vectors that are closed with respect to taking linear combinations

A set of linearly independent vectors that span the linear space

The linear space generated by taking linear combinations of a set of vectors

A basis made up of vectors that have all entries equal to zero except one

The number of elements of any one of the bases of the linear space

Two subspaces are complementary if their direct sum equals the whole space

The sum of two subspaces whose intersection contains only the zero vector

## Matrix rank and inversion

The dimension of the linear space spanned by the columns or rows of the matrix

Multiplying matrices is equivalent to taking linear combinations of their rows and columns

Multivariate generalization of the concept of reciprocal of a number

This lecture note presents some useful facts about the rank of the product of two matrices

A device that helps to invert and factorize block matrices

Formulae for computing how changes in a matrix affect its inverse

## Linear maps

Functions that preserve vector addition and scalar multiplication

Vectors containing the coefficients of the representation in terms of a basis

Linear transformations that map a space into itself

Each linear map is associated to a unique matrix that transforms coordinates

The set of vectors belonging to the domain that are mapped into the zero vector

The composition of two linear transformations is itself linear

Learn how to classify maps based on their kernel and range

The subset of the codomain formed by all the values taken by the map

Learn what happens to coordinate vectors when you switch to a different basis

The dimension of the domain of a linear map equals the sum of the dimensions of its kernel and range

The matrix of a linear operator which projects vectors onto a subspace

## Systems of linear equations

Systems of linear equations having the same set of solutions

Systems of linear equations can be written compactly and easily studied with matrices

A compact way to represent systems of linear equations

Elementary operations used in matrix algebra to transform a linear system into an equivalent system

The main algorithm used to reduce linear systems to row echelon form

Systems of linear equations having this form can be easily solved with the back-substitution algorithm

The standard algorithm used to transform linear systems to reduced row echelon form

Echelon form in which the basic columns are vectors of the standard basis

A system of equations in which the vector of constants is non-zero

A system of equations in which the vector of constants is zero

Operations that allow us to transform a linear system arranged horizontally into an equivalent system

## Special matrices and equivalence

A matrix that has all entries below (or above) the main diagonal equal to zero

A matrix used to perform multiple interchanges of rows and columns

A matrix whose off-diagonal entries are all equal to zero

A matrix obtained by performing an elementary operation on an identity matrix

How to write a matrix as a product of a lower and an upper triangular matrix

How elementary row operations generate equivalence classes

## Complex vectors and inner products

Taking both the transpose and the complex conjugate of a matrix is very common in matrix algebra

Basic facts and definitions about matrices whose entries are complex numbers

The norm of a vector generalizes the concept of length to abstract spaces

A generalization of the concept of dot product to abstract vector spaces

A procedure used in matrix algebra to create sets of orthonormal vectors

A basis whose vectors are orthogonal and have unit norm

A=QR where Q has orthonormal columns and R is upper triangular

A complex matrix whose columns form an orthonormal set

A special case of oblique projection that gives the closest vector in the subspace

The subspace formed by all the vectors that are orthogonal to a given set

A unitary matrix often used to transform another matrix into a simpler one

The ranges and kernels of a matrix and its transpose are pairwise orthogonal complements

An orthogonal matrix that can be used to perform equivalent transformations

## Determinants

A number telling us how the associated linear transformation scales volumes

A concept that pops up in the definition of determinant of a matrix

Discover several properties enjoyed by the determinant of a matrix

Determinants of elementary matrices enjoy some special properties

A formula for easily computing the determinant of a matrix

Rules about the determinants of block matrices are very useful

The trace of a matrix is the sum of the entries on its main diagonal

## Polynomials

Dividend equals divisor times quotient plus remainder, achieved with the Division Algorithm

These lecture notes summarize some facts about polynomials that are important in matrix algebra

The greatest common divisor of polynomials has properties similar to the gcd of integers

## Eigenvalues and eigenvectors

The polynomial whose roots are the eigenvalues of a matrix

Linear transformations scale up or down the sides of certain parallelograms but do not change their angles

Eigenvectors corresponding to distinct eigenvalues are linearly independent

The multiplicity of a repeated eigenvalue and the dimension of its eigenspace

Similar matrices have the same rank, trace, determinant and eigenvalues

Eigenvalues and eigenvectors possess several useful properties that are also easy to derive

Any matrix is unitarily similar to an upper triangular matrix

Transformation of a matrix into another similar matrix that is diagonal

A full-rank matrix whose eigenvalues are all strictly positive

A matrix that commutes with its conjugate transpose and is unitarily diagonalizable

Write a matrix as a product of a unitary, a diagonal and another unitary matrix

This lecture explains how to factorize a matrix into a lower triangular matrix and its conjugate transpose

A subspace that is mapped into itself by a linear operator

Decomposition Conditions on A Properties of matrices
A = LU No row interchanges for REF L lower triangular, U upper triangular
PA = LU No conditions P permutation, L and U lower and upper triangular
A = QR Full-rank Q unitary, R upper with diagonal entries > 0
A = PDP-1 (Diagonalization) No defective eigenvalues D diagonal, P invertible
A = QTQ* (Schur) No conditions T upper triangular, Q unitary
A = LL* (Choleski) Positive definite L lower triangular with diagonal entries > 0
A = USV* (Singular Value) No conditions U and V unitary, S diagonal with entries >= 0
A = PJP-1 (Jordan) No conditions J in Jordan form, P invertible

## Matrix polynomials

A certain power of a matrix can be used to decompose a space of vectors

Discover what happens to the null and column spaces of a matrix when you raise it to integer powers

If you transform the characteristic polynomial into a matrix polynomial, you get the zero matrix

Matrix powers can be used to construct polynomials, similarly to the scalar case

These lecture notes explain the most important application of the minimal polynomial

The annihilating polynomial having the lowest possible degree

Nilpotent matrices generate strings of linearly independent vectors

A matrix that becomes equal to the zero matrix if raised to a sufficiently high power

## Jordan form

A string of generalized eigenvectors ending with an ordinary eigenvector

A vector that can be used to complete a basis of eigenvectors when the matrix is defective

Any matrix is similar to an almost diagonal matrix, said to be in Jordan form ## Kronecker products and vectorizations

This lecture note presents several uselful properties of the Kronecker product

A big matrix that contains all the products of the entries of two matrices

A permutation matrix used to transpose vectorizations and commute Kronecker products

An operator that transforms any matrix into a column vector

## Matrix functions

How to apply scalar functions such as the exponential to square matrices

## Applications

A workhorse of engineering and a useful application of matrix algebra

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Glossary entries
Share