Matrix multiplication

This lecture introduces matrix multiplication, one of the basic algebraic operations that can be performed on matrices.

Table of contents

Dot product
Matrix product
Motivation
Properties of matrix multiplication
Other properties
1. Transpose of a product
Solved exercises

Dot product

Before defining matrix multiplication, we need to introduce the concept of dot product of two vectors.

Definition Let $A$ be a $1 \times K$ row vector and $B$ a $K \times 1$ column vector. Denote their entries by $A_{1}, \dots, A_{K}$ and by $B_{1}, \dots, B_{K}$ , respectively. Then, their dot product $A B$ is

A B = k = 1 \sum K A_{k} B_{k}

Note that in the above definition the order of the product matters, that is $A B$ is not the same as $B A$ , because the first vector ( $A$ ) needs to be a row vector, and the second one ( $B$ ) needs to be a column vector. Furthermore, the dot product is defined only if $A$ and $B$ have the same number of entries ( $K$ ).

Example Let $A$ be a $1 \times 3$ vector defined by

A = [215]

and $B$ a $3 \times 1$ vector defined by

B = 402

Their dot product $A B$ is

A B = 2 \cdot 4 + 1 \cdot 0 + 5 \cdot 2 = 8 + 0 + 10 = 18

Matrix product

We are now ready to define matrix product.

Definition Let $A$ be a $K \times L$ matrix and $B$ a $L \times M$ matrix. Then, their product $A B$ is a $K \times M$ matrix whose $(k, m)$ -th entry is equal to the dot product between the $k$ -th row of $A$ and the $m$ -th column of $B$ , for $1 \leq k \leq K$ and $1 \leq m \leq M$ .

In other words, the $(k, m)$ -th entry of $A B$ is

(A B)_{k m} = l = 1 \sum L A_{k l} B_{l m}

Note that the order of the product matters, that is $A B$ is not the same as $B A$ . Furthermore, the number of columns of $A$ needs to be equal to the number of rows of $B$ (in which case the two matrices are said to be conformable for the multiplication $A B$ ).

The next diagram summarizes the dimensions involved in matrix multiplication:

Dimension of the product matrix: K \times M K \times L L Need to be equal \times M

Example Define the $2 \times 3$ matrix

A = 012130

and the $3 \times 2$ matrix

B = 350010

They are conformable for the multiplication $A B$ because the number of columns of $A$ is equal to the number of rows of $B$ . The dimension of the matrix $A B$ is $2 \times 2$ . The product is

A B = 0 \cdot 3 + 2 \cdot 5 + 3 \cdot 0 1 \cdot 3 + 1 \cdot 5 + 0 \cdot 0 0 \cdot 0 + 2 \cdot 1 + 3 \cdot 0 1 \cdot 0 + 1 \cdot 1 + 0 \cdot 0 = 10821

where, for example, the $(2, 1)$ -th entry of $A B$ has been obtained from the dot product of the second row of $A$ with the first column of $B$ :

[110] 350 = 1 \cdot 3 + 1 \cdot 5 + 0 \cdot 0 = 8

Motivation

Why is matrix multiplication defined in this way? There are many possible answers to this question, but the simplest one has to do with the need of obtaining a simple matrix representation for systems of linear equations. The next example shows how.

Example Consider the following system of two equations in two unknowns:

⎩ ⎨ ⎧ 2 x_{1} + 3 x_{2} = 1 x_{1} + x_{2} = 0

This can be represented in matrix form as

A x = b

where the matrix of coefficients is

A = 2131

the vector of unknowns is

x = x_{1} x_{2}

and the vector of constants is

b = 10

You can easily check that the two ways of writing the system of equations are equivalent by performing the matrix multiplication

A x = 2131 x_{1} x_{2} = 2 x_{1} + 3 x_{2} x_{1} + x_{2}

Another reason why matrix multiplication is defined in the manner shown above is that it allows to easily deal with input-output systems in which given outputs can be obtained from fixed combinations of inputs.

Example A factory can produce two goods, denoted by $O_{1}$ and $O_{2}$ , using different combinations of two inputs, $I_{1}$ and $I_{2}$ . In particular, $2$ units of $I_{1}$ and $1$ unit of $I_{2}$ are needed to produce a unit of $O_{1}$ , and $1$ unit of $I_{1}$ and $3$ units of $I_{2}$ are needed to produce a unit of $O_{2}$ . This information can be summarized by the input-output matrix

A = 2113

where the two rows correspond to the two outputs and the two columns correspond to the two inputs. Each unit of $I_{1}$ costs $2$ dollars, and each unit of $I_{2}$ costs $1$ dollar. This information can be summarized by the vector of prices

x = 21

In order to find the costs of producing the two outputs, it suffices to perform the following matrix multiplication

A x = 211321 = 2 \cdot 2 + 1 \cdot 1 2 \cdot 1 + 1 \cdot 3 = 55

So, both outputs have a production cost of $5$ dollars.

Properties of matrix multiplication

As we have already said, unlike multiplication of real numbers, matrix multiplication does not enjoy the commutative property, that is, $A B$ is not the same as $B A$ . However, some of the properties enjoyed by multiplication of real numbers are also enjoyed by matrix multiplication.

Proposition (distributive property) Matrix multiplication is distributive with respect to matrix addition, that is,

(A + B) C C (A + B) = A C + B C = C A + C B

for any matrices $A$ , $B$ and $C$ such that the above multiplications and additions are meaningfully defined.

Proof

Let us start with the product

(A + B) C

Let $A$ and $B$ be $K \times L$ matrices, and $C$ an $L \times M$ matrix. Denote a generic $(k, l)$ -th element of the matrix $A + B$ by $(A + B)_{k l}$ , and a generic $(k, m)$ -th element of the product between $(A + B)$ and $C$ by $[(A + B) C]_{k m}$ . By the definitions of matrix addition and matrix multiplication, we have that

^{A}^{B}^{C} [(A + B) C]_{k m} = l = 1 \sum L (A + B)_{k l} C_{l m} = l = 1 \sum L (A_{k l} + B_{k l}) C_{l m} = l = 1 \sum L A_{k l} C_{l m} + l = 1 \sum L B_{k l} C_{l m} = (A C)_{k m} + (B C)_{k m}

where: in steps $A$ and $C$ we have used the definition of matrix multiplication; in step $B$ we have used the definition of matrix addition. This holds for any $(k, m)$ -th element of the matrix. Therefore, we have that

(A + B) C = A C + B C

With an almost identical argument it is possible to prove that

C (A + B) = C A + C B

Proposition (associative property) Matrix multiplication is associative, that is,

A (B C) = (A B) C

for any matrices $A$ , $B$ and $C$ such that the above multiplications are meaningfully defined.

Proof

Suppose $A$ has dimension $K \times L$ , $B$ has dimension $L \times M$ , and $C$ has dimension $M \times N$ . Associativity holds because a generic $(k, n)$ -th element of the matrix $A (B C)$ is

^{A}^{B}^{C}^{D} [A (B C)]_{k n} = l = 1 \sum L A_{k l} (B C)_{l n} = l = 1 \sum L A_{k l} m = 1 \sum M B_{l m} C_{mn} = l = 1 \sum L m = 1 \sum M A_{k l} B_{l m} C_{mn} = m = 1 \sum M l = 1 \sum L A_{k l} B_{l m} C_{mn} = m = 1 \sum M (l = 1 \sum L A_{k l} B_{l m}) C_{mn} = m = 1 \sum M (A B)_{k m} C_{mn} = [(A B) C]_{k n}

where we have used the definition of product between $A$ and $B C$ (step $A$ ), between $B$ and $C$ (step $B$ ), between $A$ and $B$ (step $C$ ), between $A B$ and $C$ (step $D$ ).

Other properties

Other properties of matrix products are listed here.

Transpose of a product

Proposition Let $A$ be a $K \times L$ matrix and $B$ a $L \times M$ matrix. Let $A^{⊤}$ and $B^{⊤}$ be their transposes. Then,

(A B)^{⊤} = B^{⊤} A^{⊤}

Proof

The $(k, m)$ -th entry of $A B$ is the dot product of the $k$ -th row of $A$ and the $m$ -th column of $B$ :

(A B)_{k m} = l = 1 \sum L A_{k l} B_{l m}

By the definition of matrix transpose, the latter is equal to the $(m, k)$ -th entry of $(A B)^{⊤}$ :

((A B)^{⊤})_{mk} = l = 1 \sum L A_{k l} B_{l m}

The $(m, k)$ -th entry $B^{⊤} A^{⊤}$ is the dot product of the $m$ -th row of $B^{⊤}$ and the $k$ -th column of $A^{⊤}$ :

(B^{⊤} A^{⊤})_{mk} = l = 1 \sum L (B^{⊤})_{m l} (A^{⊤})_{l k}

Since the $m$ -th row of $B^{⊤}$ is equal to the $m$ -th column of $B$ , and the $k$ -th column of $A^{⊤}$ is equal to the $k$ -th row of $A$ , we have

(B^{⊤} A^{⊤})_{mk} = l = 1 \sum L (B^{⊤})_{m l} (A^{⊤})_{l k} = l = 1 \sum L B_{l m} A_{k l}

Thus,

(B^{⊤} A^{⊤})_{mk} = l = 1 \sum L A_{k l} B_{l m} = ((A B)^{⊤})_{mk}

for any $m$ and $k$ . Therefore,

(A B)^{⊤} = B^{⊤} A^{⊤}

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Define a $3 \times 3$ matrix

A = 210200101

and a $3 \times 2$ matrix

B = 501030

Compute the product $A B$ .

Solution

The dimensions involved in this multiplication are summarized in the following diagram:

Dimension of the product matrix: 3 \times 2 3 \times 33 Need to be equal \times 2

Thus, $A B$ is a $3 \times 2$ matrix such that for each $1 \leq k \leq 3$ and $1 \leq m \leq 2$ , the $(k, m)$ -th element of $A B$ is equal to the dot product between the $k$ -th row of $A$ and the $m$ -th row of $B$ :

A B = 210200101501030 = 2 \cdot 5 + 2 \cdot 0 + 1 \cdot 1 1 \cdot 5 + 0 \cdot 0 + 0 \cdot 1 0 \cdot 5 + 0 \cdot 0 + 1 \cdot 1 2 \cdot 0 + 2 \cdot 3 + 1 \cdot 0 1 \cdot 0 + 0 \cdot 3 + 0 \cdot 0 0 \cdot 0 + 0 \cdot 3 + 1 \cdot 0 = 1151600

Exercise 2

Given the matrices $A$ and $B$ defined above, compute the product $B A$ .

Solution

The matrices $A$ and $B$ are not conformable for the multiplication $B A$ because the number of columns of $B$ is not equal to the number of rows of $A$ . Therefore, multiplication cannot be carried out.

Exercise 3

Define a $2 \times 1$ column vector

A = 21

and a $1 \times 3$ row vector

B = [310]

Compute the product $A B$ .

Solution

The dimensions involved in this multiplication are summarized in the following diagram:

Dimension of the product matrix: 2 \times 3 2 \times 11 Need to be equal \times 3

Thus, $A B$ is a $2 \times 3$ matrix. It is computed as follows:

A B = 21 [310] = 2 \cdot 3 1 \cdot 3 2 \cdot 1 1 \cdot 1 2 \cdot 0 1 \cdot 0 = 632100

Note that each element of $A B$ is the product of a row of $A$ with a column of $B$ . But the rows of $A$ are scalars, because $A$ is a column vector, and the columns of $B$ are also scalars, because $B$ is a row vector. As a consequence, each entry of $A B$ is obtained as the product of two scalars.

How to cite

Please cite as:

Taboga, Marco (2021). "Matrix multiplication", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/matrix-multiplication.