MTH 215 Chapter 6

Section 6.1: Inner Product, Length, Orthogonality

Being familiar already with length, distance, and perpendicularity in $\R^2$ and $\R^3$, we define these terms in arbitrary vector spaces.

Note that you can define many inner products on a given vector space! We will stick to the “standard” ones that are most often used.

If $\vec{u}$ and $\vec{v}$ are vectors in $\R^n$, then the (standard) inner product (sometimes called dot product) between them is defined to be \[ \vec{u} \bullet \vec{v} \eq \vec{u}^T \vec{v} \eq u_1 v_1 + u_2 v_2 + \dots + u_n v_n .\]

Compute the inner product between the two vectors:

$\vec{u} = \mat{r} 1 \\ -2 \rix$, $\vec{v} = \mat{r} 2 \\ -5 \rix$.
$\vec{x} = \mat{r} 2 \\ -5 \\ -1 \rix$, $\vec{y} = \mat{r} 3 \\ 2 \\ -3 \rix$

Let $\vec{u}, \vec{v}, \vec{w}$ be vectors in $\R^n $ and $c\in \R $, with inner product $\vec{u} \bullet \vec{v} = \vec{u}^T \vec{v}$. The following properties hold:

$\vec{u} \bullet \vec{v} = \vec{v} \bullet \vec{u}$
$(\vec{u} + \vec{v}) \bullet \vec{w} = \vec{u} \bullet \vec{w} + \vec{v} \bullet \vec{w}$
$(c \vec{u}) \bullet \vec{v} = c (\vec{u} \bullet \vec{v}) = \vec{u} \bullet (c \vec{v})$
$\vec{u} \bullet \vec{u} \ge 0$, and $\vec{u} \bullet \vec{u} = 0$ if and only if $\vec{u} = \vec{0}$

Warning! The last property does not say that $\vec{u} \bullet \vec{v} \ge 0$ for a general $\vec{v}$!

Inner products allow us to define length in a general vector space.

The length or norm of a vector $\vec{v}$ in $\R^n$ is defined by \[ \norm{\vec{v}} = \sqrt{\vec{v} \bullet \vec{v}} = \sqrt{v_1^2 + v_2^2+ \dots + v_n^2} .\]

The vector v with components x and y in the second quadrant of the xy-plane. The vector points north-west, and has its components indicated on the coordinate axes.

Compute the norms of $\vec{v} = \mat{r} 1 \\ -2 \\ 2 \\ 0 \rix$ and $\vec{x} = \mat{r} 3 \\ 2 \rix$.

For any scalar $c \in \R$, then the length of $c \vec{v}$ is $\abs{c}$ times the length of $\vec{v}$: \[ \norm{c \vec{v}} \eq \abs{c} \, \norm{\vec{v}} .\] We often like to work with vectors of norm $1$, called unit vectors, which can easily be done:

Given $\vec{v} \in \R^n$, the unit vector in the direction of $v$ is the vector:

For $\vec{v} = \mat{r} 1 \\ -2 \\ 2 \\ 0 \rix$ and $\vec{x} = \mat{r} 3 \\ 2 \rix$, we found $\norm{\vec{v}} = 3$ and $\norm{\vec{x}} = \sqrt{13}$. Then the unit vectors in the directions of $\vec{v}$ and $\vec{x}$ are:

The vector x given by the coordinates (3,2) in the xy-plane. The unit vector corresponding to x is also displayed in red.

Now, we are ready to describe how “close” one vector is to another.

For $\vec{u}$ and $\vec{v}$ in $\R^n$, the distance between $\vec{u}$ and $\vec{v}$, denoted $\dist(\vec{u}, \vec{v})$, is the norm of the vector $\vec{u} - \vec{v}$: \[ \dist(\vec{u}, \vec{v}) \eq \norm{\vec{u} - \vec{v}} .\]

The vectors u = (7,1) and v = (3,2) on a graph of the xy-plane, shown in black, with the distance vector u-v=(4,-1) shown in green, and the red distance length shown.

We can also use inner products and norms to define angles between vectors. In $\R^2$, this is entirely consistent with what we know from geometry. Take the last example as a reference:

The vectors u = (7,1) and v = (3,2) on a graph of the xy-plane, shown in black, with the angle theta between u and v of approximately 30 degrees, and an indication of the norm of the distance between u and v.

Recall the Law of Cosines: \[ \norm{\vec{u} - \vec{v}} ^2 = \norm{\vec{u}}^2 + \norm{\vec{v}}^2 - 2 \norm{\vec{u}} \, \norm{\vec{v}} \cos (\theta) .\]

In the case of $\R^2$, if $\vec{u} = (u_1, u_2)$ and $\vec{v}_2 = (v_1, v_2)$, then \begin{align*} \norm{\vec{u}} \, \norm{\vec{v}} \cos (\theta ) & \eq \hspace{25em} \\ & \eq \\ & \eq \\ & \eq \\ \end{align*}

For any two vectors $\vec{u}$ and $\vec{v}$ in $\R^n $, the angle $\theta $ between them is given by \[ \cos (\theta ) \eq \hspace{20em} \]

For the example above, recall that $\vec{u} = (7,1)$ and $\vec{v} = (3,2)$. Therefore: \[ \cos (\theta ) \eq \]

Our last topic is one of orthogonality—vectors that are perpendicular.

Recall the angle between two vectors $\vec{u}$ and $\vec{v}$ in $\R^n$ is given by $\cos (\theta )= \dfrac{\vec{u} \bullet \vec{v}}{\norm{\vec{u}} \, \norm{\vec{v}}}$.

If $\theta =90^\circ $, then ...

Two vectors $\vec{u}$ and $\vec{v}$ in $\R^n $ are said to be orthogonal (perpendicular) if $\vec{u} \bullet \vec{v} = 0$.

Are the following vectors orthogonal?

$\vec{u} = (2,4)$ and $\vec{v} = (-2,1)$.
$\vec{u} = (1,1)$ and $\vec{v} = (-3,-3)$.
$\vec{u} = (3,2,-5,0)$ and $\vec{v} = (-4,1,-2,6)$.

The Pythagorean Theorem extends to higher dimensions using the inner product in $\R^n$.

Two vectors $\vec{u}$ and $\vec{v}$ in $\R^n $ are orthogonal if and only if $\norm{\vec{u} + \vec{v}}^2 = \norm{\vec{u}}^2 + \norm{\vec{v}}^2$.

Orthogonal vectors turn out to be relevant in regard to the fundamental subspaces of a matrix. One can show the following (which we saw in Homework 4), and will justify the picture at the end of the Chapter 4 notes.

Let $A$ be any $m\times n$ matrix. Suppose $\vec{x}$ is in $\Col (A)$ and $\vec{y}$ is in $\Null (A^T)$. Show that $\vec{x} \bullet \vec{y} = 0$.

Now let $\vec{x}$ be in $\Row(A)$ and $\vec{y}$ be in $\Null (A)$. Show that $\vec{x} \bullet \vec{y} = 0$.

Let $A$ be a symmetric $n\times n$ matrix. Suppose $A \vec{x} = \lambda \vec{x}$ and $A \vec{y} = \mu \vec{y}$ where $\lambda \neq \mu $ are eigenvalues of $A$.

What can be said about $\vec{x}$ and $\vec{y}$? About $A$?

Inner products, norms, and orthogonality can be defined in vector spaces other than $\R^n $. There are requirements the inner product function must satisfy (not be discussed here).

When doing so, we will write $\vect{\vec{x}, \vec{y}}$ instead of $\vec{x} \bullet \vec{y}$ because the latter is strictly reserved for $\R^n$. However, we still have $\norm{\vec{x}} = \sqrt{\vect{\vec{x}, \vec{x}} }$, and also two “vectors” $\vec{x}$ and $\vec{y}$ are orthogonal if $\vect{\vec{x}, \vec{y}} =0$, just as in $\R^n $.

For example, in $\cP_2$ (vector space of polynomials of degree at most $2$), one can define \begin{equation}\label{equ-p2-int} \vect{f,g} \eq \int_{0}^{1} f(x) g(x) \: dx \qquad \text{ for } f,g \in \cP_2. \end{equation}

A different inner product on $\cP_2$ is \begin{equation}\label{equ-p2-ip} \vect{f, g} \eq f(-1)g(-1) + f(0)g(0) + f(1)g(1) \qquad \text{ for } f,g \in \cP_2 .\end{equation}

Suppose $f(x) = x^2 + 2x - 3$ and $g(x) = 2x^2-2x + 1$. Using the inner product defined in equation \eqref{equ-p2-ip}, then:

$\vect{f,g} = $
$\norm{f} = $

Compute $\vect{f,g}$ for $f(x)=x$ and $g(x)=1-2x^2$ with the inner product defined in \eqref{equ-p2-ip}. (If you know calculus, try the inner product in \eqref{equ-p2-int}).