Matrix Multiplication - The Inner and Outer Products

The Inner and Outer Products

Given two column vectors a and b, the Euclidean inner product and outer product are the simplest special cases of the matrix product, by transposing the column vectors into row vectors.

The inner product

is a column vector multiplied on the left by a row vector:

More explicitly,

\mathbf{a}\cdot \mathbf{b} = \begin{pmatrix}a_1 & a_2 & \cdots & a_n\end{pmatrix} \begin{pmatrix}b_1 \\ b_2 \\ \vdots \\ b_n\end{pmatrix} = a_1b_1+a_2b_2+\cdots+a_nb_n = \sum_{i=1}^n a_ib_i
The outer product

is a row vector multiplied on the left by a column vector:

where

\mathbf{a}\mathbf{b}^\mathrm{T} = \begin{pmatrix}a_1 \\ a_2 \\ \vdots \\ a_n\end{pmatrix} \begin{pmatrix}b_1 & b_2 & \cdots & b_n\end{pmatrix} = \begin{pmatrix} a_1 b_1 & a_1 b_2 & \cdots & a_1 b_n \\ a_2 b_1 & a_2 b_2 & \cdots & a_2 b_n \\ \vdots & \vdots & \ddots & \vdots \\ a_n b_1 & a_n b_2 & \cdots & a_n b_n \\ \end{pmatrix}.


Matrix product (in terms of inner product)

Suppose that the first n×m matrix A is decomposed into its row vectors ai, and the second m×p matrix B into its column vectors bi:

\mathbf{A} = \begin{pmatrix} A_{1 1} & A_{1 2} & \cdots & A_{1 m} \\ A_{2 1} & A_{2 2} & \cdots & A_{2 m} \\ \vdots & \vdots & \ddots & \vdots \\ A_{n 1} & A_{n 2} & \cdots & A_{n m} \end{pmatrix} = \begin{pmatrix} \mathbf{a}_1 \\ \mathbf{a}_2 \\ \vdots \\ \mathbf{a}_n \end{pmatrix},\quad \mathbf{B} = \begin{pmatrix} B_{1 1} & B_{1 2} & \cdots & B_{1 p} \\ B_{2 1} & B_{2 2} & \cdots & B_{2 p} \\ \vdots & \vdots & \ddots & \vdots \\ B_{m 1} & B_{m 2} & \cdots & B_{m p} \end{pmatrix} = \begin{pmatrix} \mathbf{b}_1 & \mathbf{b}_2 & \cdots & \mathbf{b}_p \end{pmatrix}

where

The entries in the introduction were given by:

\mathbf{AB} = \begin{pmatrix} \mathbf{a}_1 \\ \mathbf{a}_2 \\ \vdots \\ \mathbf{a}_n \end{pmatrix} \begin{pmatrix} \mathbf{b}_1 & \mathbf{b}_2 & \dots & \mathbf{b}_p \end{pmatrix} = \begin{pmatrix} (\mathbf{a}_1 \cdot \mathbf{b}_1) & (\mathbf{a}_1 \cdot \mathbf{b}_2) & \dots & (\mathbf{a}_1 \cdot \mathbf{b}_p) \\ (\mathbf{a}_2 \cdot \mathbf{b}_1) & (\mathbf{a}_2 \cdot \mathbf{b}_2) & \dots & (\mathbf{a}_2 \cdot \mathbf{b}_p) \\ \vdots & \vdots & \ddots & \vdots \\ (\mathbf{a}_n \cdot \mathbf{b}_1) & (\mathbf{a}_n \cdot \mathbf{b}_2) & \dots & (\mathbf{a}_n \cdot \mathbf{b}_p) \end{pmatrix}


It is also possible to express a matrix product in terms of concatenations of products of matrices and row or column vectors:

\mathbf{AB} = \begin{pmatrix} \mathbf{a}_1 \\ \mathbf{a}_2 \\ \vdots \\ \mathbf{a}_n \end{pmatrix} \begin{pmatrix} \mathbf{b}_1 & \mathbf{b}_2 & \dots & \mathbf{b}_p \end{pmatrix} = \begin{pmatrix} \mathbf{A}\mathbf{b}_1 & \mathbf{A}\mathbf{b}_2 & \dots & \mathbf{A}\mathbf{b}_p \end{pmatrix} = \begin{pmatrix} \mathbf{a}_1\mathbf{B} \\ \mathbf{a}_2\mathbf{B}\\ \vdots\\ \mathbf{a}_n\mathbf{B} \end{pmatrix}

These decompositions are particularly useful for matrices that are envisioned as concatenations of particular types of row vectors or column vectors, e.g. orthogonal matrices (whose rows and columns are unit vectors orthogonal to each other) and Markov matrices (whose rows or columns sum to 1).

Matrix product (in terms of outer product)

An alternative method results when the decomposition is done the other way around, i.e. the first matrix A is decomposed into column vectors and the second matrix B into row vectors :

\mathbf{AB} = \begin{pmatrix} \mathbf{\bar a}_1 & \mathbf{\bar a}_2 & \cdots & \mathbf{\bar a}_m \end{pmatrix} \begin{pmatrix} \mathbf{\bar b}_1 \\ \mathbf{\bar b}_2 \\ \vdots \\ \mathbf{\bar b}_m \end{pmatrix} = \mathbf{\bar a}_1 \otimes \mathbf{\bar b}_1 + \mathbf{\bar a}_2 \otimes \mathbf{\bar b}_2 + \cdots + \mathbf{\bar a}_m \otimes \mathbf{\bar b}_m = \sum_{i=1}^m \mathbf{\bar a}_i \otimes \mathbf{\bar b}_i

where this time

This method emphasizes the effect of individual column/row pairs on the result, which is a useful point of view with e.g. covariance matrices, where each such pair corresponds to the effect of a single sample point.

Read more about this topic:  Matrix Multiplication

Famous quotes containing the words outer and/or products:

    “No one knows better than children how much they need the authority that protects, that sets the outer limits of behavior with known and prescribed consequences. As one little boy expressed it to his mother, “You care what I do.””
    —Leontine Young (20th century)

    “All that is told of the sea has a fabulous sound to an inhabitant of the land, and all its products have a certain fabulous quality, as if they belonged to another planet, from seaweed to a sailor’s yarn, or a fish story. In this element the animal and vegetable kingdoms meet and are strangely mingled.”
    —Henry David Thoreau (1817–1862)