Skip to main content

Mathematical Scholars Calculus 3

Gabriel Kerr

Section 3.2 Geometry II

Now that we have a way to discuss length and angles, a whole new set of questions arises. For example, when we put a coordinate chart on an inner product space, we may want this chart to have axes that are perpendicular to each other. We may also want the coordinates to agree with some unit length so that, for example, the vector with coordinates \((0, 1, 0)\) has length \(1\text{.}\) Perhaps you will be pleasantly surprised that such notions have names and many of the details concerning them have been worked out completely. We will cover some of them in this section.
We start with a nice result which tells us how far a little geometry can take us.

Proof.

Suppose we have a linear relation
\begin{equation*} \mb{0} = a_1 \mb{v}_1 + \cdots + a_n \mb{v}_n . \end{equation*}
We may pair both sides with \(\mb{v}_i\) to obtain
\begin{align*} 0 \amp = \left\langle \mb{0} , \mb{v}_i \right\rangle , \\ \amp = \left\langle a_1 \mb{v}_1 + \cdots + a_n \mb{v}_n , \mb{v}_i \right\rangle, \\ \amp = a_1 \left\langle \mb{v}_1, \mb{v}_i \right\rangle + \cdots + a_n \left\langle \mb{v}_n , \mb{v}_i \right\rangle, \\ \amp = a_i \left\langle \mb{v}_i , \mb{v}_i \right\rangle . \end{align*}
Since \(\mb{v}_i\) are non-zero, we have that \(\left\langle \mb{v}_i , \mb{v}_i \right\rangle \gt 0\) by the positive definite property. So dividing implies \(a_i = 0\) for every \(i\) and the set of vectors is linearly independent.
Of course, we may want a little more than orthogonality of vectors.

Definition 3.2.2.

We say that the vectors \(\mb{v}_1 , \ldots, \mb{v}_n\) of an inner product space \(V\) are orthonormal if they are pairwise orthogonal unit vectors. We call \(\mathcal{B} = \{ \mb{v}_1 , \ldots, \mb{v}_n \}\) an orthonormal basis if it is a basis of orthonormal vectors.
Although I may not convince you of this, I must assert that there’s nothing in life that’s better than having a good orthonormal basis! This is means that not only do you have a basis and thus a system of coordinates for your vector space, but you have coordinates which reflect the geometry of Euclidean space (in the real setting and a little bit more in the complex setting). Let’s see a practical application of this.

Example 3.2.3. Formula for coordinates.

Recall that if \(\mathcal{B} = \{ \mb{v}_1 , \ldots, \mb{v}_n \}\) was a basis for \(V\) then we had a coordinate map
\begin{equation*} \coord{}{\mathcal{B}} : V \to K^n . \end{equation*}
This map is in fact easy to describe - it just takes a vector \(\mb{v}\) and produces the column vector
\begin{equation*} \threevec{a_1}{\vdots}{a_n} \end{equation*}
whose entries are the unique coefficients in the linear combination \(\mb{v} = a_1 \mb{v}_1 + \cdots + a_n \mb{v}_n\text{.}\) While easy to describe, there is not generally a formula for \(\coord{}{\mathcal{B}}\text{,}\) and one may need to solve a linear system to obtain the map. However, if \(V\) is an inner product space and \(\mathcal{B}\) is an orthonormal basis, we have an immediate and easy formula
\begin{equation} \coord{\mb{v}}{\mathcal{B}} = \threevec{\left\langle \mb{v} , \mb{v}_1 \right\rangle}{\vdots}{\left\langle \mb{v} , \mb{v}_n \right\rangle} \tag{3.2.1} \end{equation}
To see that this formula works, pair the difference
\begin{equation*} \mb{w} = \mb{v} - (\left\langle \mb{v} , \mb{v}_1 \right\rangle \mb{v}_1 + \cdots + \left\langle \mb{v} , \mb{v}_n \right\rangle \mb{v}_n ) \end{equation*}
with any basis vector to see
\begin{align*} \left\langle \mb{w} , \mb{v}_i \right\rangle \amp = \left\langle \mb{v} - (\left\langle \mb{v} , \mb{v}_1 \right\rangle \mb{v}_1 + \cdots + \left\langle \mb{v} , \mb{v}_n \right\rangle \mb{v}_n ) , \mb{v}_i \right\rangle,\\ \amp = \left\langle \mb{v} , \mb{v}_i \right\rangle - \left\langle \mb{v} , \mb{v}_i \right\rangle \left\langle \mb{v}_i , \mb{v}_i \right\rangle, \\ \amp = 0. \end{align*}
But \(\mb{w}\) itself is a linear combination of the \(\mb{v}_i\) and so the above equality implies \(\left\langle \mb{w} , \mb{w} \right\rangle = 0\text{.}\) But this implies that \(\mb{w} = \mb{0}\) which means the coefficients of \(\mathcal{B}\) giving \(\mb{v}\) are exactly \(\left\langle \mb{v} , \mb{v}_i \right\rangle\text{.}\)
Let us apply this result for a computational example in \(\mathbb{R}^3\) with the dot product. Consider the orthonormal basis
\begin{equation*} \mathcal{B} = \left\{ \threevec{\sqrt{2} / 2}{\sqrt{2}/ {2}}{0} , \threevec{\sqrt{2} / 2}{ -\sqrt{2}/ {2}}{0} , \threevec{0}{0}{1} \right\}. \end{equation*}
We can then write coordinates of
\begin{equation*} \mb{v} = \threevec{2}{-4}{3} \end{equation*}
simply by taking dot products
\begin{equation*} \coord{v}{\mathcal{B}} = \threevec{2 (\sqrt{2} / 2) -4 (\sqrt{2} / 2) + 0}{2 (\sqrt{2} / 2) + ( -4) (-\sqrt{2} / 2) + 0}{0 + 0 + 3} = \threevec{-\sqrt{2}}{3\sqrt{2}}{3} . \end{equation*}
Now, one might wonder whether, given a finite dimensional inner product space \(V\text{,}\) we can always obtain an orthonormal basis. Happily, we have a nice method or process. In fact, it will be better for us to perform this algorithm to first obtain an orthogonal basis, and then afterwards normalize to get an orthonormal one. Let us describe this method inductively by assuming we have been given a basis \(\mathcal{B} = \{\mb{v}_1, \ldots, \mb{v}_n\}\text{.}\)
  1. Induction Hypothesis.
    Assume we have found non-zero orthogonal vectors
    \begin{equation*} \{\mb{w}_1 , \ldots , \mb{w}_{k - 1}\} \end{equation*}
    so that
    \begin{equation*} \textnormal{span} (\{\mb{w}_1 , \ldots , \mb{w}_{k - 1}\} ) = \textnormal{span} (\{\mb{v}_1 , \ldots , \mb{v}_{k - 1}\}). \end{equation*}
  2. Induction Step.
    We define \(k\)-th vector
    \begin{equation} \mb{w}_k = \mb{v}_k - \left( \frac{\left\langle \mb{v}_k, \mb{w}_1 \right\rangle}{\left\langle \mb{w}_1 , \mb{w}_1 \right\rangle} \mb{w}_1 + \cdots + \frac{\left\langle \mb{v}_k, \mb{w}_{k - 1} \right\rangle}{\left\langle \mb{w}_{k - 1} , \mb{w}_{k - 1} \right\rangle} \mb{w}_{k - 1} \right). \tag{3.2.2} \end{equation}
This process is called the Gram-Schmidt process and we write it as a Theorem.
We now write a short script that will apply this process to a collection of \(k\) vectors. This is written just to give the orthogonal basis and leaves the normalization step.
Let’s quickly describe what’s happenning in the script. The code def applyGS(vectors): defines a function which takes input as a list of vectors. So, if you enter a list of other types, you will get an error. What follows in the indentation is the definition of the applyGS function. The first part of n = len(vector) declares \(n\) as the number of vectors in the list vectors (len is short for length). Then we create the list newVectors which is a copy of the inputed vectors and will be altered as we apply Gram-Schmidt.
The code for k in range(1,n): starts a for loop which begins at \(k = 1\) and keeps going until \(k = n\text{.}\) The for loop declares vk as the vector \(\mb{v}_k\) and creates the short list inductionWs which corresponds to the vectors \(\{\mb{w}_1, \ldots, \mb{w}_{k - 1} \}\text{.}\) The list adjustedWs scales these vectors to
\begin{equation} \left\{ - \frac{\left\langle \mb{v}_k, \mb{w}_1 \right\rangle}{\left\langle \mb{w}_1 , \mb{w}_1 \right\rangle} \mb{w}_1 , \ldots , -\frac{\left\langle \mb{v}_k, \mb{w}_{k - 1} \right\rangle}{\left\langle \mb{w}_{k - 1} , \mb{w}_{k - 1} \right\rangle} \mb{w}_{k - 1} \right\}\tag{3.2.3} \end{equation}
Then theSum adds these vectors together and wk is defined as the new \(\mb{w}_k\) as in equation (3.2.2). The final step changes the \(k\)-th vector of newVectors to wk. The code return(newVectors) tells the function to return the list \(\{\mb{w}_1, \ldots, \mb{w}_n\}\text{.}\)
After defining the function, we make a list of three vectors and evaluate the function on this list. You may alter the list by changing the vectors to vectors in any fixed dimension and adding more if desired.
Now that we know such nice bases exist, we proceed to exploit their usefulness. First, we note that we may use inner products to decompose our entire space into two subspaces. For this we need a couple of definitions.

Definition 3.2.5.

Given a vector space \(V\text{,}\) we say that two vector subspaces \(V_1\) and \(V_2\) are complementary if
\begin{align*} V_1 \cap V_2 \amp = \{ \mb{0} \}, \\ V_1 + V_2 \amp = V. \end{align*}
In this case, every vector \(\mb{v}\) can be written uniquely as the sum
\begin{equation*} \mb{v} = \mb{v}_1 + \mb{v}_2 \end{equation*}
where \(\mb{v}_1\) is in \(V_1\) and \(\mb{v}_2\) is in \(V_2\text{.}\) We write
\begin{equation*} V = V_1 \oplus V_2 \end{equation*}
and say \(V\) is a direct sum of \(V_1\) and \(V_2\text{.}\)
It is not hard to obtain examples of direct sums. Indeed, the following lemma says that one can always find a complement to a proper subspace.

Proof.

Since \(W\) is finite dimensional, so is \(U\text{.}\) If \(\{\mb{u}_1, \ldots, \mb{u}_k \}\) is a basis of \(U\text{,}\) one can extend it by adding vectors \(\mb{v}_1 , \ldots, \mb{v}_{n - k}\) to make a basis of \(W\text{.}\) But it is not too hard to show that taking \(V\) to be the span of \(\mb{v}_1 , \ldots, \mb{v}_{n - k}\) gives a complementary subspace with dimension equation in the lemma.

Example 3.2.7. Complementary lines in \(\mathbb{R}^2\).

If we take the line
\begin{equation*} U = \left\{ \lambda \twovec{1}{1} : t \in \mathbb{R} \right\} \subset \mathbb{R}^2 \end{equation*}
then any line through the origin that does not have slope \(1\) is complementary to \(U\text{.}\) Observe that this shows that complementary subspaces are not unique.
When the vector space is an inner product space, the previous lemma has a natural solution.

Definition 3.2.8.

If \(V\) is an inner product space and \(U\) is a subspace, we say that the orthogonal complement to \(U\) is the vector subspace
\begin{equation*} U^\perp = \left\{ \mb{v} : \left\langle \mb{v} , \mb{u} \right\rangle = 0 \text{ for all } \mb{u} \in U \right\} \end{equation*}
You will show that this is a vector subspace in the exercises.

Proof.

Let \(\mb{u}_1, \ldots, \mb{u}_k\) be an orthonormal basis for \(U\) and \(\mb{u}_1^\prime, \ldots , \mb{u}_l^\prime\) an orthonormal basis for \(U^\perp\text{.}\) We claim then that \(\mathcal{B} = \{\mb{u}_1, \ldots, \mb{u}_k, \mb{u}_1^\prime, \ldots , \mb{u}_l^\prime\}\) is a basis for \(V\text{.}\) First, one sees that it must be linearly independent by noting that it is an orthogonal set of vectors and applying Lemma 3.2.1.
To see that it spans \(V\text{,}\) suppose that \(\mb{v}\) is not in the span of \(\mb{u}_1, \ldots, \mb{u}_k\text{.}\) Then we may apply Gram-Schmidt to \(\mb{u}_1, \ldots, \mb{u}_k, \mb{v}\) and obtain a vector
\begin{equation*} \mb{w} = \mb{v} - (\left\langle\mb{v}, \mb{u}_1 \right\rangle \mb{u}_1 + \cdots + \left\langle\mb{v}, \mb{u}_k \right\rangle \mb{u}_k ). \end{equation*}
But pairing \(\mb{w}\) with any basis element \(\mb{u}_i\) gives zero which implies by linearity that \(\mb{w}\) is in \(U^\perp\text{.}\) This, in turn says that \(\mb{w}\) is a linear combination of \(\mb{u}_1^\prime, \ldots , \mb{u}_l^\prime\) which shows that \(\mb{v}\) is in the span of \(\mathcal{B}\text{.}\)
Thus, writing \(\mb{v}\) as a linear combination of the two bases yield the unique sum \(\mb{u} + \mb{u}^\prime\text{.}\)
In fact, this proposition gives us the existence of a linear projection.

Definition 3.2.10.

Let \(V\) be an inner product space and \(U\) a subspace. Given a vector \(\mb{v}\text{,}\) the projection of \(\mb{v}\) to \(U\text{,}\) denoted \(\textnormal{proj}_U (\mb{v})\) is the vector in \(U\) which is closest to \(\mb{v}\text{.}\)
This definition may seem a bit cryptic, but we make it plain with a proposition generalizing Exercise 3.1.4.4.

Proof.

To see the first equation, we note that Lemma 3.2.9 has already given us a unique decomposition \(\mb{v} = \mb{u} + \mb{u}^\prime\text{.}\) Given any vector \(\mb{\tilde{u}} \in U\) we observe that the square of the distance from \(\mb{v}\) to \(\mb{\tilde{u}}\) is
\begin{align*} \| \mb{v} - \mb{\tilde{u}} \|^2 \amp = \|\mb{u} + \mb{u}^\prime -\mb{\tilde{u}} \|^2, \\ \amp = \| ( \mb{u} - \mb{\tilde{u}} ) + \mb{u}^\prime \|^2, \\ \amp = \| \mb{u} - \mb{\tilde{u}} \|^2 + \| \mb{u}^\prime \|^2 . \end{align*}
Here we used the fact that \(\left\langle \mb{u} - \mb{\tilde{u}},\mb{u}^\prime \right\rangle = 0\) because both \(\mb{u}\) and \(\mb{\tilde{u}}\) are in \(U\) while \(\mb{u}^\prime\) is in \(U^\perp\text{.}\) Clearly, if \(\mb{u} \ne \mb{\tilde{u}}\) then this distance squared is greater than \(\|\mb{u}^\prime \|^2\text{.}\) Thus \(\mb{u}\) minimizes the distance from \(U\) to \(\mb{v}\) and is the projection. Examining the proof of Lemma 3.2.9 we see that \(\mb{u}\) is given explicitly as the linear combination in equation (3.2.4).

Exercises Exercises

1.

Let \([a,b] = [0, 2\pi]\) and consider the Hilbert space \(L^2 ([0, 2\pi])\) given in Example 3.1.13. Show that the set
\begin{equation*} \{ \ldots, e^{-2it}, e^{-it}, 1, e^{it}, e^{2it}, \ldots \} \end{equation*}
is in fact an orthonormal set of vectors. In fact, this set makes up what is known as a Hilbert basis (which is a good infinite dimensional version of a basis).

2.

Let
\begin{equation*} \mathcal{B} = \left\{ \threevec{\sqrt{3} / 3}{\sqrt{3}/ {3}}{\sqrt{3}/ {3}} , \threevec{\sqrt{2} / 2}{ -\sqrt{2}/ {2}}{0} , \threevec{\sqrt{6}/6}{\sqrt{6}/6}{-\sqrt{6}/3} \right\} \end{equation*}
in the inner product space \(\mathbb{R}^3\) with the dot product.
(a)
Show that \(\mathcal{B}\) is an orthonormal basis.
(b)
If
\begin{equation*} \mb{v} = \threevec{1}{2}{1} \end{equation*}
what is \(\coord{\mb{v}}{\mathcal{B}}\text{?}\)

3.

Apply the Gram-Schmidt algorithm to obtain orthogonal bases for the subspaces spanned by the following collections of vectors. Give an orthonormal bases afterwards.
(a)
The vectors
\begin{equation*} \twovec{1}{2} , \twovec{2}{3}. \end{equation*}
(b)
The two vectors spanning the plane \(W\) in \(\mathbb{R}^4\)
\begin{equation*} \left[ \begin{matrix} 1 \\ -1 \\ 1 \\ -1 \end{matrix} \right] , \left[ \begin{matrix} 2 \\ 1 \\ 1 \\ 1 \end{matrix} \right] . \end{equation*}

4.

Use the Sage function to learn about applying Gram-Schmidt.
(a)
Choose a random basis of four vectors in Euclidean \(\mathbb{R}^4\) which are linearly independent but not orthogonal. Use the function applyGS() in the Sage cell above equation (3.2.3) to apply Gram-Schmidt to your list and write the result.
(b)
Change the third vector in your original list to the sum of the first two and evaluate applyGS() to the new list. What happened? Explain why you obtained this response.
Hint.
Use applyGS() on the list consisting only of your first three vectors. What does this tell you?

5.

Given a vector subspace \(U\) of an inner product space \(V\text{,}\) show that the orthogonal complement \(U^\perp\) of \(U\) is in fact a vector subspace.

6.

Let
\begin{equation*} \mb{v} = \left[ \begin{matrix} 1 \\ 1 \\ 1 \\ 0 \end{matrix} \right] \end{equation*}
Find \(\textnormal{proj}_W (\mb{v} )\) where \(W\) was given in part (b) of Exercise 3.2.3.

7.

Find a basis for \(W^\perp\) where \(W\) was given in part (b) of Exercise 3.2.3.
Hint.
Solve the matrix equation \(A \mb{x} = \mb{0}\) where \(A\) has rows equal to the transpose of the two vectors in part (b)
PrevTopNext