Skip to main content
Logo image

Section 2.4 Linear independence

In the previous section, questions about the existence of solutions of a linear system led to the concept of the span of a set of vectors. In particular, the span of a set of vectors v1,v2,…,vn is the set of vectors b for which a solution to the linear system [v1v2…vn] x=b exists.
In this section, we turn to the uniqueness of solutions of a linear system, the second of our two fundamental questions. This will lead us to the concept of linear independence.

Preview Activity 2.4.1.

Let’s begin by looking at some sets of vectors in R3. As we saw in the previous section, the span of a set of vectors in R3 will be either a line, a plane, or R3 itself.
  1. Consider the following vectors in R3:
    v1=[0−12],v2=[31−1],v3=[201].
    Describe the span of these vectors, Span{v1,v2,v3}, as a line, a plane, or R3.
  2. Now consider the set of vectors:
    w1=[0−12],w2=[31−1],w3=[301].
    Describe the span of these vectors, Span{w1,w2,w3}, as a line, a plane, or R3.
  3. Show that the vector w3 is a linear combination of w1 and w2 by finding weights such that
    w3=cw1+dw2.
  4. Explain why any linear combination of w1, w2, and w3,
    c1w1+c2w2+c3w3
    can be written as a linear combination of w1 and w2.
  5. Explain why
    Span{w1,w2,w3}=Span{w1,w2}.

Subsection 2.4.1 Linear dependence

We have seen examples where the span of a set of three vectors in R3 is R3 and other examples where the span of three vectors is a plane. We would like to understand the difference between these two situations.

Example 2.4.1.

Let’s consider the set of three vectors in R3:
v1=[220],   v2=[11−1],   v3=[−101].
Forming the associated matrix gives
[v1v2v3]=[21−12100−11]∼[100010001].
Because there is a pivot position in every row, Proposition 2.3.14 tells us that Span{v1,v2,v3}=R3.

Example 2.4.2.

Now let’s consider the set of three vectors:
w1=[220],   w2=[11−1],   w3=[−5−51].
Forming the associated matrix gives
[w1w2w3]=[21−521−50−11]∼[10−201−1000].
Since the last row does not have a pivot position, we know that the span of these vectors is not R3 but is instead a plane.
In fact, we can say more if we shift our perspective slightly and view this as an augmented matrix:
[w1w2w3]=[21−521−50−11]∼[10−201−1000].
In this way, we see that w3=−2w1−w2, which enables us to rewrite any linear combination of these three vectors:
c1w1+c2w2+c3w3=c1w1+c2w2+c3(−2w1−w2)=(c1−2c3)w1+(c2−c3)w2.
In other words, any linear combination of w1, w2, and w3 may be written as a linear combination using only the vectors w1 and w2. Since the span of a set of vectors is simply the set of their linear combinations, this shows that
Span{w1,w2,w3}=Span{w1,w2}.
As a result, adding the vector w3 to the set of vectors w1 and w2 does not change the span.
Before exploring this type of behavior more generally, let’s think about it from a geometric point of view. Suppose that we begin with the two vectors v1 and v2 in Example 2.4.1. The span of these two vectors is a plane in R3, as seen on the left of Figure 2.4.3.
Figure 2.4.3. The span of the vectors v1, v2, and v3.
Because the vector v3 is not a linear combination of v1 and v2, it provides a direction to move that is independent of v1 and v2. Adding this third vector v3 therefore forms a set whose span is R3, as seen on the right of Figure 2.4.3.
Similarly, the span of the vectors w1 and w2 in Example 2.4.2 is also a plane. However, the third vector w3 is a linear combination of w1 and w2, which means that it already lies in the plane formed by w1 and w2, as seen in Figure 2.4.4. Since we can already move in this direction using just w1 and w2, adding w3 to the set does not change the span. As a result, it remains a plane.
Figure 2.4.4. The span of the vectors w1, w2, and w3.
What distinguishes these two examples is whether one of the vectors is a linear combination of the others, an observation that leads to the following definition.

Definition 2.4.5.

A set of vectors is called linearly dependent if one of the vectors is a linear combination of the others. Otherwise, the set of vectors is called linearly independent.
For the sake of completeness, we say that a set of vectors containing only one nonzero vector is linearly independent.

Subsection 2.4.2 How to recognize linear dependence

Activity 2.4.2.

We would like to develop a means to detect when a set of vectors is linearly dependent. This activity will point the way.
  1. Suppose we have five vectors in R4 that form the columns of a matrix having reduced row echelon form
    [v1v2v3v4v5]∼[10−102012030001−100000].
    Is it possible to write one of the vectors v1,v2,…,v5 as a linear combination of the others? If so, show explicitly how one vector appears as a linear combination of some of the other vectors. Is this set of vectors linearly dependent or independent?
  2. Suppose we have another set of three vectors in R4 that form the columns of a matrix having reduced row echelon form
    [w1w2w3]∼[100010001000].
    Is it possible to write one of these vectors w1, w2, w3 as a linear combination of the others? If so, show explicitly how one vector appears as a linear combination of some of the other vectors. Is this set of vectors linearly dependent or independent?
  3. By looking at the pivot positions, how can you determine whether the columns of a matrix are linearly dependent or independent?
  4. If one vector in a set is the zero vector 0, can the set of vectors be linearly independent?
  5. Suppose a set of vectors in R10 has twelve vectors. Is it possible for this set to be linearly independent?
By now, we should expect that the pivot positions play an important role in determining whether the columns of a matrix are linearly dependent. For instance, suppose we have four vectors and their associated matrix
[v1v2v3v4]∼[102001−3000010000].
Since the third column does not contain a pivot position, let’s just focus on the first three columns and view them as an augmented matrix:
[v1v2v3]∼[10201−3000000].
This says that
v3=2v1−3v2,
which tells us that the set of vectors v1,v2,v3,v4 is linearly dependent. Moreover, we see that
Span{v1,v2,v3,v4}=Span{v1,v2,v4}.
More generally, the same reasoning implies that a set of vectors is linearly dependent if the associated matrix has a column without a pivot position. Indeed, as illustrated here, a vector corresponding to a column without a pivot position can be expressed as a linear combination of the vectors whose columns do contain pivot positions.
Suppose instead that the matrix associated to a set of vectors has a pivot position in every column.
[w1w2w3w4]∼[10000100001000010000].
Viewing this as an augmented matrix again, we see that the linear system is inconsistent since there is a pivot in the rightmost column, which means that w4 cannot be expressed as a linear combination of the other vectors. Similarly, w3 cannot be expressed as a linear combination of w1 and w2. In fact, none of the vectors can be written as a linear combination of the others so this set of vectors is linearly independent.
The following proposition summarizes these findings.
This condition imposes a constraint on how many vectors we can have in a linearly independent set. Here is an example of the reduced row echelon form of a matrix whose columns form a set of three linearly independent vectors in R5:
[100010001000000].
Notice that there are at least as many rows as columns, which must be the case if every column is to have a pivot position.
More generally, if v1,v2,…,vn is a linearly independent set of vectors in Rm, the associated matrix must have a pivot position in every column. Since every row contains at most one pivot position, the number of columns can be no greater than the number of rows. This means that the number of vectors in a linearly independent set can be no greater than the number of dimensions.
This says, for instance, that any linearly independent set of vectors in R3 can contain no more three vectors. We usually imagine three independent directions, such as up/down, front/back, left/right, in our three-dimensional world. This proposition tells us that there can be no more independent directions.
The proposition above says that a set of vectors in Rm that is linear independent has at most m vectors. By comparison, Proposition 2.3.15 says that a set of vectors whose span is Rm has at least m vectors.

Subsection 2.4.3 Homogeneous equations

If A is a matrix, we call the equation Ax=0 a homogeneous equation. As we’ll see, the uniqueness of solutions to this equation reflects on the linear independence of the columns of A.

Activity 2.4.3. Linear independence and homogeneous equations.

  1. Explain why the homogeneous equation Ax=0 is consistent no matter the matrix A.
  2. Consider the matrix
    A=[320−10−2211]
    whose columns we denote by v1, v2, and v3. Describe the solution space of the homogeneous equation Ax=0 using a parametric description, if appropriate.
  3. Find a nonzero solution to the homogeneous equation and use it to find weights c1, c2, and c3 such that
    c1v1+c2v2+c3v3=0.
  4. Use the equation you found in the previous part to write one of the vectors as a linear combination of the others.
  5. Are the vectors v1, v2, and v3 linearly dependent or independent?
This activity shows how the solution space of the homogeneous equation Ax=0 indicates whether the columns of A are linearly dependent or independent. First, we know that the equation Ax=0 always has at least one solution, the vector x=0. Any other solution is a nonzero solution.

Example 2.4.8.

Let’s consider the vectors
v1=[2−410],   v2=[113−2],   v3=[3−34−2]
and their associated matrix A=[v1v2v3].
The homogeneous equation Ax=0 has the associated augmented matrix
[2130−41−3013400−2−20]∼[1010011000000000].
Therefore, A has a column without a pivot position, which tells us that the vectors v1, v2, and v3 are linearly dependent. However, we can also see this fact in another way.
The reduced row echelon matrix tells us that the homogeneous equation has a free variable so that there must be infinitely many solutions. In particular, we have
x1=−x3x2=−x3
so the solutions have the form
x=[x1x2x3]=[−x3−x3x3]=x3[−1−11].
If we choose x3=1, then we obtain the nonzero solution to the homogeneous equation x=[−1−11], which implies that
A[−1−11]=[v1v2v3][−1−11]=−v1−v2+v3=0.
In other words,
−v1−v2+v3=0v3=v1+v2.
Because v3 is a linear combination of v1 and v2, we know that this set of vectors is linearly dependent.
As this example demonstrates, there are many ways we can view the question of linear independence, some of which are recorded in the following proposition.

Subsection 2.4.4 Summary

This section developed the concept of linear dependence of a set of vectors. More specifically, we saw that:
  • A set of vectors is linearly dependent if one of the vectors is a linear combination of the others.
  • A set of vectors is linearly independent if and only if the vectors form a matrix that has a pivot position in every column.
  • A set of linearly independent vectors in Rm contains no more than m vectors.
  • The columns of the matrix A are linearly dependent if the homogeneous equation Ax=0 has a nonzero solution.
  • A set of vectors v1,v2,…,vn is linearly dependent if there are weights c1,c2,…,cn, not all of which are zero, such that
    c1v1+c2v2+…+cnvn=0.
At the beginning of the section, we said that this concept addressed the second of our two fundamental questions concerning the uniqueness of solutions to a linear system. It is worth comparing the results of this section with those of the previous one so that the parallels between them become clear.
As usual, we will write a matrix as a collection of vectors,
A=[v1v2…vn].
Table 2.4.10. Span and Linear Independence
Span Linear independence
A vector b is in the span of a set of vectors if it is a linear combination of those vectors.
A set of vectors is linearly dependent if one of the vectors is a linear combination of the others.
A vector b is in the span of v1,v2,…,vn if there exists a solution to Ax=b.
The vectors v1,v2,…,vn are linearly independent if x=0 is the unique solution to Ax=0.
The columns of an m×n matrix span Rm if the matrix has a pivot position in every row.
The columns of a matrix are linearly independent if the matrix has a pivot position in every column.
A set of vectors that span Rm has at least m vectors.
A set of linearly independent vectors in Rm has at most m vectors.

Exercises 2.4.5 Exercises

1.

Consider the set of vectors
v1=[121],v2=[013],v3=[23−1],v4=[−24−1].
  1. Explain why this set of vectors is linearly dependent.
  2. Write one of the vectors as a linear combination of the others.
  3. Find weights c1, c2, c3, and c4, not all of which are zero, such that
    c1v1+c2v2+c3v3+c4v4=0.
  4. Suppose A=[v1v2v3v4]. Find a nonzero solution to the homogenous equation Ax=0.

2.

Consider the vectors
v1=[2−10],v2=[121],v3=[2−23].
  1. Are these vectors linearly independent or linearly dependent?
  2. Describe the Span{v1,v2,v3}.
  3. Suppose that b is a vector in R3. Explain why we can guarantee that b may be written as a linear combination of v1, v2, and v3.
  4. Suppose that b is a vector in R3. In how many ways can b be written as a linear combination of v1, v2, and v3?

3.

Respond to the following questions and provide a justification for your responses.
  1. If the vectors v1 and v2 form a linearly dependent set, must one vector be a scalar multiple of the other?
  2. Suppose that v1,v2,…,vn is a linearly independent set of vectors. What can you say about the linear independence or dependence of a subset of these vectors?
  3. Suppose v1,v2,…,vn is a linearly independent set of vectors that form the columns of a matrix A. If the equation Ax=b is inconsistent, what can you say about the linear independence or dependence of the set of vectors v1,v2,…,vn,b?

4.

Determine whether the following statements are true or false and provide a justification for your response.
  1. If v1,v2,…,vn are linearly dependent, then one vector is a scalar multiple of one of the others.
  2. If v1,v2,…,v10 are vectors in R5, then the set of vectors is linearly dependent.
  3. If v1,v2,…,v5 are vectors in R10, then the set of vectors is linearly independent.
  4. Suppose we have a set of vectors v1,v2,…,vn and that v2 is a scalar multiple of v1. Then the set is linearly dependent.
  5. Suppose that v1,v2,…,vn are linearly independent and form the columns of a matrix A. If Ax=b is consistent, then there is exactly one solution.

5.

Suppose we have a set of vectors v1,v2,v3,v4 in R5 that satisfy the relationship:
2v1−v2+3v3+v4=0
and suppose that A is the matrix A=[v1v2v3v4].
  1. Find a nonzero solution to the equation Ax=0.
  2. Explain why the matrix A has a column without a pivot position.
  3. Write one of the vectors as a linear combination of the others.
  4. Explain why the set of vectors is linearly dependent.

6.

Suppose that v1,v2,…,vn is a set of vectors in R27 that form the columns of a matrix A.
  1. Suppose that the vectors span R27. What can you say about the number of vectors n in this set?
  2. Suppose instead that the vectors are linearly independent. What can you say about the number of vectors n in this set?
  3. Suppose that the vectors are both linearly independent and span R27. What can you say about the number of vectors in the set?
  4. Assume that the vectors are both linearly independent and span R27. Given a vector b in R27, what can you say about the solution space to the equation Ax=b?

7.

Given below are some descriptions of sets of vectors that form the columns of a matrix A. For each description, give a possible reduced row echelon form for A or indicate why there is no set of vectors satisfying the description by stating why the required reduced row echelon matrix cannot exist.
  1. A set of 4 linearly independent vectors in R5.
  2. A set of 4 linearly independent vectors in R4.
  3. A set of 3 vectors whose span is R4.
  4. A set of 5 linearly independent vectors in R3.
  5. A set of 5 vectors whose span is R4.

8.

When we explored matrix multiplication in Section 2.2, we saw that some properties that are true for real numbers are not true for matrices. This exercise will investigate that in some more depth.
  1. Suppose that A and B are two matrices and that AB=0. If B≠0, what can you say about the linear independence of the columns of A?
  2. Suppose that we have matrices A, B and C such that AB=AC. We have seen that we cannot generally conclude that B=C. If we assume additionally that A is a matrix whose columns are linearly independent, explain why B=C. You may wish to begin by rewriting the equation AB=AC as AB−AC=A(B−C)=0.

9.

Suppose that k is an unknown parameter and consider the set of vectors
v1=[201],v2=[4−2−1],v3=[02k].
  1. For what values of k is the set of vectors linearly dependent?
  2. For what values of k does the set of vectors span R3?

10.

Given a set of linearly dependent vectors, we can eliminate some of the vectors to create a smaller, linearly independent set of vectors.
  1. Suppose that w is a linear combination of the vectors v1 and v2. Explain why Span{v1,v2,w}=Span{v1,v2}.
  2. Consider the vectors
    v1=[2−10],v2=[121],v3=[−262],v4=[7−11].
    Write one of the vectors as a linear combination of the others. Find a set of three vectors whose span is the same as Span{v1,v2,v3,v4}.
  3. Are the three vectors you are left with linearly independent? If not, express one of the vectors as a linear combination of the others and find a set of two vectors whose span is the same as Span{v1,v2,v3,v4}.
  4. Give a geometric description of Span{v1,v2,v3,v4} in R3 as we did in Section 2.3.
You have attempted 1 of 1 activities on this page.