---

---

Defining Matrices

Let’s start by taking a look at the following two matrices.

$$A= \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} ,\,\, B= \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6\\ \end{bmatrix}$$

Matrices are measures rows X columns. $A$ is a $3\times2$ matrix and $B$ is a $2\times3$ matrix.

Elements in a matrix are also defined by their row and column. For instance, $A_{ij}$ corresponds to row $i$ and column $j$. Looking above, $A_{21} = 3$ and $B_{21} = 4$.

Vectors

A vector is an $nx1$ matrix.

$$c= \begin{bmatrix} 1\\ 3\\ 5\\ \end{bmatrix} ,\, d= \begin{bmatrix} 2\\ 4\\ 6\\ \end{bmatrix}$$

By convention, vectors are defined as lowercase letters and matrices use uppercase.

Matrix Addition and Subtraction

We can only add matrices or vectors that have the same dimensions.

$$\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} + \begin{bmatrix} 10 & 10 \\ 10 & 10 \\ 10 & 10 \\ \end{bmatrix} = \begin{bmatrix} 11 & 12 \\ 13 & 14 \\ 15 & 16 \\ \end{bmatrix}$$

As you can see we add the values based on their row and column. Row 1 column 1 is combined with row 1 column 1 from the second matrix.

Subtraction works in the same way.

$$\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} - \begin{bmatrix} 10 & 10 \\ 10 & 10 \\ 10 & 10 \\ \end{bmatrix} = \begin{bmatrix} -9 & -8 \\ -7 & -6 \\ -5 & -4 \\ \end{bmatrix}$$

An operation like the following would not work because the dimensions are not the same.

$$\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} - \begin{bmatrix} 10 & 10 & 10\\ 10 & 10 & 10\\ 10 & 10 & 10\\ \end{bmatrix}$$

Matrix - Scalar Multiplication

For scalar multiplication we multiply every element by the scalar. Scalar multiplication always returns the same dimensions.

$$3 \cdot \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} = \begin{bmatrix} 3 \cdot 1 & 3 \cdot 2 \\ 3 \cdot 3 & 3 \cdot 4 \\ 3 \cdot 5 & 3 \cdot 6 \\ \end{bmatrix} = \begin{bmatrix} 3 & 6 \\ 6 & 12 \\ 15 & 18 \\ \end{bmatrix}$$

Matrix - Vector Multiplication

For valid matrix multiplication we need the number of columns in the first matrix to equal the number of rows in the second. This is the same for vectors and matrices as a vector is just a matrix with one column.

The following would not be valid because there are more rows in the second matrix than columns in the first matrix.

$$\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} \cdot \begin{bmatrix} 1\\ 2\\ 3\\ \end{bmatrix}$$

However, if we remove a row from the second matrix it would work.

$$\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} \cdot \begin{bmatrix} 1\\ 2\\ \end{bmatrix} = \begin{bmatrix} 5\\ 11\\ 17\\ \end{bmatrix}$$

What this is saying is for the first column in the resulting matrix, we are grabbing 1 of column 1 and combining it with 2 of column 2.

If we expand our multiplier to $2\times2$ our resulting matrix will be $3\times2$.

$$\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \\ \end{bmatrix} \cdot \begin{bmatrix} 1 & 2\\ 2 & 4\\ \end{bmatrix} = \begin{bmatrix} 5 & 10\\ 11 & 22\\ 17 & 34\\ \end{bmatrix}$$

Paying attention to the second column in the result, we see that we combined 2 of the first column and 4 of the second column.

The multiplier simply tells us how many of each row to grab for a specific column.

Matrix Multiplication Properties

$A \times B \neq B \times A$ however $A \times ( B \times C ) = (A \times B) \times C$

Examples

$$\begin{bmatrix} 1 & 1\\ 0 & 0\\ \end{bmatrix} \times \begin{bmatrix} 0 & 0\\ 2 & 0\\ \end{bmatrix} = \begin{bmatrix} 2 & 0\\ 0 & 0\\ \end{bmatrix}$$ $$\begin{bmatrix} 0 & 0\\ 2 & 0\\ \end{bmatrix} \times \begin{bmatrix} 1 & 1\\ 0 & 0\\ \end{bmatrix} = \begin{bmatrix} 0 & 0\\ 2 & 2\\ \end{bmatrix}$$

Matrix Multiplication Applied

More info about the hypothesis function

Let’s say we have a hypothesis function $h_\theta(x) = -40 + .25x$. We are going to assume this function is the best fit for determining housing prices given a square footage.

In the following matrix the first column is contains all identities, or ones, and the second column contains house sizes.

$$\begin{bmatrix} 1 & 2104 \\ 1 & 1416 \\ 1 & 1534 \\ 1 & 852 \\ \end{bmatrix}$$

This second matrix represents our hypothesis function.

$$\begin{bmatrix} -40\\ .25 \end{bmatrix}$$

What happens is the $1$ in the first column keeps $-40$ as is because it it the equivalent of multiplying by $1$. The second column is multiplied by .25 and is combined with $-40$.

$$\begin{bmatrix} 1 & 2104 \\ 1 & 1416 \\ 1 & 1534 \\ 1 & 852 \\ \end{bmatrix} \cdot \begin{bmatrix} -40\\ .25 \end{bmatrix} = \begin{bmatrix} h_\theta(2104) \\ h_\theta(1416) \\ h_\theta(1534) \\ h_\theta(852) \\ \end{bmatrix}$$

We could also test multiple hypothesis at once by adding more columns to the multiplier like so.

$$\begin{bmatrix} 1 & 2104 \\ 1 & 1416 \\ 1 & 1534 \\ 1 & 852 \\ \end{bmatrix} \cdot \begin{bmatrix} -40 & 200\\ .25 & .1 \end{bmatrix}$$

Each resulting column is a different hypothesis.