Subsection 2.5.1 Independence and Dimension
For that we need the following definition.
Definition 2.5.1.
Let \(V\) be a vector space over \(K\text{.}\) We say that the vectors \(\mb{v}_1, \ldots, \mb{v}_n\) are linearly independent if the equality
\begin{equation*}
a_1 \mb{v}_1 + \cdots + a_n \mb{v}_n = \mb{0}
\end{equation*}
implies that \(a_1, \ldots , a_n\) are all zero.
The equation in
Definition 2.5.1 is called a
linear relation. So an alternative way of saying that the vectors are linearly independent is that there is no non-trivial linear relation. Yet another way of expressing linear independence is given in the following proposition which the reader can check.
Proposition 2.5.2.
A set of vectors is linearly independent if and only if they form a basis of their span.
One practical reason to want a linearly independent set of vectors is to parameterize (or ‘give coordinates’ to) their span. Indeed, if \(\mathcal{A} = \{\mb{v}_1 , \ldots, \mb{v}_n \}\) is an ordered set of vectors in \(V\text{,}\) then define the function
\begin{equation*}
\chart{}{\mathcal{A}} : K^n \to V
\end{equation*}
by
\begin{equation}
\left[ \begin{matrix} a_1 \\ \vdots \\ a_n \end{matrix} \right]_{\mathcal{A}} = a_1 \mb{v}_1 + \cdots + a_n \mb{v}_n. \tag{2.5.1}
\end{equation}
Then a third way of expressing linear independence can be given as follows.
Proposition 2.5.3.
The set of vectors \(\mathcal{A}\) is linearly independent if and only if \(\chart{}{\mathcal{A}}\) is one-to-one.
Note that if \(\mathcal{A}\) is also a basis of \(V\) (i.e. it spans all of \(V\)), then \(\chart{}{\mathcal{A}}\) is the inverse function to \(\coord{}{\mathcal{A}}\text{.}\)
There are instances when it is nice to be able to parameterize lines, planes and higher dimensional subspaces in a given vector space, and \(\chart{}{\mathcal{A}}\) can often be used for this purpose. But to do this in \(V = K^m\text{,}\) one must first see that a given set of column vectors is linearly independent.
Proposition 2.5.4.
Suppose \(\mb{c}_1, \ldots, \mb{c}_n\) are column vectors in \(K^m\text{.}\) They are linearly independent if and only if the matrix
\begin{equation*}
A = \left[ \begin{matrix} | \amp | \amp \amp | \\ \mb{c}_1 \amp \mb{c}_2 \amp \cdots \amp \mb{c}_n \\ | \amp | \amp \amp | \end{matrix} \right]
\end{equation*}
has no free columns.
Proof.
To see this, just recall from
Example 2.2.12 the fact that any linear combination of the
\(\mb{c}_i\)’s can be written as the product
\begin{equation*}
\left[ \begin{matrix} | \amp | \amp \amp | \\ \mb{c}_1 \amp \mb{c}_2 \amp \cdots \amp \mb{c}_n \\ | \amp | \amp \amp | \end{matrix} \right] \left[ \begin{matrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{matrix} \right] = a_1 \mb{c}_1 + a_2 \mb{c}_2 + \cdots a_n \mb{c}_n.
\end{equation*}
In particular, the numbers \(a_1, \ldots, a_n\) give a linear relation
\begin{equation*}
a_1 \mb{c}_1 + a_2 \mb{c}_2 + \cdots a_n \mb{c}_n = \mb{0}
\end{equation*}
if and only if they solve the matrix equation
\begin{equation*}
A \mb{x} = \mb{0}.
\end{equation*}
But
Theorem 2.4.7 Case (2) says that the solution is unique if and only if there are no free variables of the reduced row echelon equation
\(A^\prime \mb{x} = \mb{0}\text{.}\) This implies there are no free columns of
\(A\text{.}\)
Proposition 2.5.4, along with row reduction, gives a computational way to check linear independence.
Example 2.5.5. Testing linear independence by solving a matrix equation.
Let us see whether the vectors
\begin{equation*}
\threevec{1}{3}{-1} , \threevec{0}{2}{1}, \threevec{2}{4}{-3}
\end{equation*}
are linearly independent or not. We compute the reduced row echelon form to see if every column is basic:
\begin{align*}
\left[ \begin{matrix} 1 \amp 0 \amp 2 \\ 3 \amp 2 \amp 4 \\ -1 \amp 1 \amp -3 \end{matrix} \right] \amp \stackrel{\mb{r}_2 - 3 \mb{r}_1}{\longrightarrow} \left[ \begin{matrix} 1 \amp 0 \amp 2 \\ 0 \amp 2 \amp -2 \\ -1 \amp 1 \amp -3 \end{matrix} \right], \\ \amp \stackrel{\mb{r}_3 + \mb{r}_1}{\longrightarrow} \left[ \begin{matrix} 1 \amp 0 \amp 2 \\ 0 \amp 2 \amp -2 \\ 0 \amp 1 \amp -1 \end{matrix} \right],\\
\amp \stackrel{1/2 \mb{r}_2}{\longrightarrow} \left[ \begin{matrix} 1 \amp 0 \amp 2 \\ 0 \amp 1 \amp -1 \\ 0 \amp 1 \amp -1 \end{matrix} \right],\\
\amp \stackrel{\mb{r}_3 - \mb{r}_2}{\longrightarrow} \left[ \begin{matrix} 1 \amp 0 \amp 2 \\ 0 \amp 1 \amp -1 \\ 0 \amp 0 \amp 0 \end{matrix} \right].
\end{align*}
We can see that the third column gives a free variable so our set is not linearly independent (also called linearly dependent).
However, by simply forgetting about the third vector (or column in the computation) we see that the first two vectors
\begin{equation*}
\threevec{1}{3}{-1} , \threevec{0}{2}{1}
\end{equation*}
are linearly independent. Being linearly independent depends heavily on the collection of vectors you are considering.
Corollary 2.5.6.
If \(\mb{c}_1, \ldots, \mb{c}_n\) are linearly independent vectors in \(K^m\text{,}\) then \(n \leq m\text{.}\)
Proof.
\begin{equation*}
A = \left[ \begin{matrix} | \amp | \amp \amp | \\ \mb{c}_1 \amp \mb{c}_2 \amp \cdots \amp \mb{c}_n \\ | \amp | \amp \amp | \end{matrix} \right].
\end{equation*}
Suppose \(A^\prime\) is the matrix \(A\) in reduced row echelon form. Then \(A^\prime\) has a leading coefficient in every column (otherwise that column corresponds to a free variable). But this gives at least \(n\) leading coefficients for \(n\) unique rows. So the number of rows \(m\) must be at least \(n\) or, in other words, \(n \leq m\text{.}\)
Another important application of
Proposition 2.5.4 leads directly to the definition of dimension.
Corollary 2.5.7.
If \(V\) is a vector space over \(K\text{,}\) then any two bases of \(V\) have the same number of elements.
Proof.
Suppose
\(V\) has bases
\(\mathcal{B} = \{\mb{v}_1, \ldots , \mb{v}_n \}\) and
\(\mathcal{C} = \{ \mb{w}_1, \ldots, \mb{w}_m \}\text{.}\) Then by exercise
Exercise 2.3.4, we have that there are isomorphisms
\begin{align*}
\coord{}{\mathcal{B}} : V \amp \to K^n \amp \coord{}{\mathcal{C}} : V \amp \to K^m.
\end{align*}
Now, since \(\mathcal{B}\) is a basis, it is a linearly independent set. And since \(\coord{}{\mathcal{C}}\) is one-to-one, the vectors \(\coord{\mb{v}_1}{\mathcal{C}}, \ldots , \coord{\mb{v}_n}{\mathcal{C}}\) must also be linearly independent (why?). But by the previous corollary, this implies that \(n \leq m\text{.}\) Using the same argument of the vectors \(\coord{\mb{w}_1}{\mathcal{B}}, \ldots , \coord{\mb{w}_m}{\mathcal{B}}\) in \(K^n\) shows that \(m \leq n\text{.}\) Together this gives \(m = n\text{.}\)
This tells us that, while there are infinitely many bases of a vector space, all of them have the same number of vectors in them. This allows us to make the following definition.
Definition 2.5.8.
Given a vector space \(V\) over \(K\text{,}\) we say that the dimension of \(V\) is \(n\) if there is a basis of \(V\) with \(n\) elements. This is denoted \(\dim_K (V) = n\text{.}\) If \(V\) has no finite basis, we say that \(V\) is infinite dimensional.
We record a nice fact that uses our knowledge of linear independence.
Corollary 2.5.9.
If \(W\) is a proper vector subspace of a finite dimensional space \(V\) then \(W\) is finite dimensional and \(\dim W \lt \dim V\text{.}\)
Proof.
Suppose
\(V\) is
\(n\) dimensional. Then by
Corollary 2.5.6 and the argument above, any linearly independent set in
\(V\) has at most
\(n\) vectors. Let
\(\mathcal{A} = \{\mb{w}_1, \ldots, \mb{w}_k\}\) be a set of linearly independent vectors in
\(W\) of maximal size. Then we contend that
\(\mathcal{A}\) is a basis of
\(W\text{.}\) If not, then there is a
\(\mb{w} \in W\) which is not in the span of
\(\mathcal{A}\text{.}\) But then we see that
\(\{\mb{w} , \mb{w}_1, \ldots, \mb{w}_k\}\) is also linearly independent (why?). This contradiction shows that
\(W\) must have a basis.
Now, as
\(W\) is a proper subspace of
\(V\text{,}\) there is a vector
\(\mb{v}\) of
\(V\) which is not in
\(W\text{.}\) But then, for the same reason as above, the set
\(\{\mb{v}, \mb{w}_1, \ldots, \mb{w}_k\}\) must be linearly independent. By
Corollary 2.6.4, this means
\(k + 1 \leq n\) which implies
\(k \lt n\text{.}\)
Instead of considering linear independence of rows and columns, we can focus on the span of the rows and/or columns.
Definition 2.5.10.
Given an \(m \times n\) matrix \(A\) with entries in \(K\text{,}\)
the column space of \(A\) is the span of the columns of \(A\) in \(K^m\text{.}\)
the row space of \(A\) is the span of the rows of \(A\) in \(K^n\text{.}\)
Note that Example
Example 2.2.12 shows that the column space is equal to the image of the linear transformation obtained by left multiplying a column vector in
\(K^n\) by
\(A\) while the row space is the image of right multiplying a row vector in
\(K^m\text{.}\) The following proposition gives bases for both of these spaces.
Proposition 2.5.11.
Let \(A\) be an \(m \times n\) matrix and \(A^\prime\) its reduced row echelon form. Suppose \(\mathcal{C} = \{ \mb{c}_{i_1}, \ldots, \mb{c}_{i_s}\}\) are the basic columns of \(A\text{,}\) and \(\mathcal{R} = \{\mb{r}^\prime_1, \ldots , \mb{r}^\prime_s \}\) are the first \(s\) rows of \(A^\prime\text{.}\) Then \(\mathcal{C}\) is a basis for the column space of \(A\) and \(\mathcal{R}\) is a basis for the row space of \(A\text{.}\)
Proof.
To see that
\(\mathcal{C}\) is a basis, note that it is linearly independent by
Proposition 2.5.4. Indeed, the reduced row echelon form of the matrix with
only these columns is the same as the submatrix of
\(A^\prime\) with only these columns. In particular, it has no free columns and the cited proposition implies that
\(\mathcal{C}\) is linearly independent. On the other hand, adding any other column of
\(A\) to
\(\mathcal{C}\) fails the test in
Proposition 2.5.4 for the same reason. But that means that any other column can be written as a linear combination of the columns in
\(\mathcal{C}\) (why?). Thus
\(\mathcal{C}\) spans the column space and is linearly independent. By
Proposition 2.5.2,
\(\mathcal{C}\) is a basis for the column space.
For the row space, we note that all of the row vectors of \(\mathcal{R}\) are linear combinations of the rows in \(A\) because row operations create rows that are linear combinations of the existing rows. Also, since row operations are reversible (in other words, \(A\) can be recovered by performing row operations on \(A^\prime\)), every row of \(A\) is a linear combination of those in \(A^\prime\text{.}\) This implies the span of \(\mathcal{R}\) is the row space (why?). On the other hand, if
\begin{equation*}
a_1 \mb{r}^\prime_1 + \cdots + a_s \mb{r}^\prime_s
\end{equation*}
is a linear combination, then projecting to the \(i_1, i_2, \ldots, i_s\) basic columns, we have the vector
\begin{equation*}
\left[ \begin{matrix} a_1 \amp a_2 \amp \cdots \amp a_s \end{matrix} \right]
\end{equation*}
This is because the coordinate of
\(\mb{r}_j^\prime\) in the
\(i_j\) column is
\(1\) and is zero for all other
\(i_k\) columns (because
\(A^\prime\) is in reduced row echelon form). Thus the linear combination is the zero vector only if all
\(a_i\) are zero, meaning
\(\mathcal{R}\) is linearly independent. Again by
Proposition 2.5.2 we have that
\(\mathcal{R}\) is a basis.
We note here that the bases for the column and row spaces have the same number of vectors! This motivates the following definition.
Definition 2.5.12.
The rank of \(A\) is the dimension of the row space of \(A\text{,}\) or equivalently, the dimension of the column space of \(A\text{.}\)The student should now be able to understand some of jargin in the Sage description of vector subspaces. Indeed, the methods .row_space() and .column_space() can be used to define the row and column space of a matrix.
The ‘degree’ in this description is the dimension of the ambient space (i.e. the number of columns), and the rank is the dimension of the row space. Indeed, using the method .dimension() the following code checks that the rank of \(A\) also equals the dimension of the column space.
