Skip to main content
Logo image

Section 6.2 Orthogonal complements and the matrix transpose

We’ve now seen how the dot product enables us to determine the angle between two vectors and, more specifically, when two vectors are orthogonal. Moving forward, we will explore how the orthogonality condition simplifies many common tasks, such as expressing a vector as a linear combination of a given set of vectors.
This section introduces the notion of an orthogonal complement, the set of vectors each of which is orthogonal to a prescribed subspace. We’ll also find a way to describe dot products using matrix products, which allows us to study orthogonality using many of the tools for understanding linear systems that we developed earlier.

Preview Activity 6.2.1.

  1. Sketch the vector v=[12] on Figure 6.2.1 and one vector that is orthogonal to it.
    Figure 6.2.1. Sketch the vector v and one vector orthogonal to it.
  2. If a vector x is orthogonal to v, what do we know about the dot product vx?
  3. If we write x=[xy], use the dot product to write an equation for the vectors orthogonal to v in terms of x and y.
  4. Use this equation to sketch the set of all vectors orthogonal to v in Figure 6.2.1.
  5. Section 3.5 introduced the column space Col(A) and null space Nul(A) of a matrix A. If A is a matrix, what is the meaning of the null space Nul(A)?
  6. What is the meaning of the column space Col(A)?

Subsection 6.2.1 Orthogonal complements

The preview activity presented us with a vector v and led us through the process of describing all the vectors orthogonal to v. Notice that the set of scalar multiples of v describes a line L, a 1-dimensional subspace of R2. We then described a second line consisting of all the vectors orthogonal to v. Notice that every vector on this line is orthogonal to every vector on the line L. We call this new line the orthogonal complement of L and denote it by L. The lines L and L are illustrated on the left of Figure 6.2.2.
Figure 6.2.2. On the left is a line L and its orthogonal complement L. On the right is a plane W and its orthogonal complement W in R3.
The next definition places this example into a more general context.

Definition 6.2.3.

Given a subspace W of Rm, the orthogonal complement of W is the set of vectors in Rm each of which is orthogonal to every vector in W. We denote the orthogonal complement by W.
A typical example appears on the right of Figure 6.2.2. Here we see a plane W, a two-dimensional subspace of R3, and its orthogonal complement W, which is a line in R3.
As the next activity demonstrates, the orthogonal complement of a subspace W is itself a subspace of Rm.

Activity 6.2.2.

Suppose that w1=[102] and w2=[111] form a basis for W, a two-dimensional subspace of R3. We will find a description of the orthogonal complement W.
  1. Suppose that the vector x is orthogonal to w1. If we write x=[x1x2x3], use the fact that w1x=0 to write a linear equation for x1, x2, and x3.
  2. Suppose that x is also orthogonal to w2. In the same way, write a linear equation for x1, x2, and x3 that arises from the fact that w2x=0.
  3. If x is orthogonal to both w1 and w2, these two equations give us a linear system Bx=0 for some matrix B. Identify the matrix B and write a parametric description of the solution space to the equation Bx=0.
  4. Since w1 and w2 form a basis for the two-dimensional subspace W, any vector w in W can be written as a linear combination
    w=c1w1+c2w2.
    If x is orthogonal to both w1 and w2, use the distributive property of dot products to explain why x is orthogonal to w.
  5. Give a basis for the orthogonal complement W and state the dimension dimW.
  6. Describe (W), the orthogonal complement of W.

Example 6.2.4.

If L is the line defined by v=[123] in R3, we will describe the orthogonal complement L, the set of vectors orthogonal to L.
If x is orthogonal to L, it must be orthogonal to v so we have
vx=x12x2+3x3=0.
We can describe the solutions to this equation parametrically as
x=[x1x2x3]=[2x23x3x2x3]=x2[210]+x3[301].
Therefore, the orthogonal complement L is a plane, a two-dimensional subspace of R3, spanned by the vectors [210] and [301].

Example 6.2.5.

Suppose that W is the 2-dimensional subspace of R5 with basis
w1=[12234],w2=[24202].
We will give a description of the orthogonal complement W.
If x is in W, we know that x is orthogonal to both w1 and w2. Therefore,
w1x=x12x2+2x3+3x44x5=0w2x=2x1+4x2+2x3+0x4+2x5=0
In other words, Bx=0 where
B=[1223424202][1201200111].
The solutions may be described parametrically as
x=[x1x2x3x4x5]=x2[21000]+x4[10110]+x5[20101].
The distributive property of dot products implies that any vector that is orthogonal to both w1 and w2 is also orthogonal to any linear combination of w1 and w2 since
(c1w1+c2w2)x=c1w1x+c2w2x=0.
Therefore, W is a 3-dimensional subspace of R5 with basis
v1=[21000],v2=[10110],v3=[20101].
One may check that the vectors v1, v2, and v3 are each orthogonal to both w1 and w2.

Subsection 6.2.2 The matrix transpose

The previous activity and examples show how we can describe the orthogonal complement of a subspace as the solution set of a particular linear system. We will make this connection more explicit by defining a new matrix operation called the transpose.

Definition 6.2.6.

The transpose of the m×n matrix A is the n×m matrix AT whose rows are the columns of A.

Example 6.2.7.

If A=[43051213], then AT=[41320153]

Activity 6.2.3.

This activity illustrates how multiplying a vector by AT is related to computing dot products with the columns of A. You’ll develop a better understanding of this relationship if you compute the dot products and matrix products in this activity without using technology.
  1. If B=[341202], write the matrix BT.
  2. Suppose that
    v1=[202],v2=[112],w=[223].
    Find the dot products v1w and v2w.
  3. Now write the matrix A=[v1v2] and its transpose AT. Find the product ATw and describe how this product computes both dot products v1w and v2w.
  4. Suppose that x is a vector that is orthogonal to both v1 and v2. What does this say about the dot products v1x and v2x? What does this say about the product ATx?
  5. Use the matrix AT to give a parametric description of all the vectors x that are orthogonal to v1 and v2.
  6. Remember that Nul(AT), the null space of AT, is the solution set of the equation ATx=0. If x is a vector in Nul(AT), explain why x must be orthogonal to both v1 and v2.
  7. Remember that Col(A), the column space of A, is the set of linear combinations of the columns of A. Therefore, any vector in Col(A) can be written as c1v1+c2v2. If x is a vector in Nul(AT), explain why x is orthogonal to every vector in Col(A).
The previous activity demonstrates an important connection between the matrix transpose and dot products. More specifically, the components of the product ATx are simply the dot products of the columns of A with x. We will make frequent use of this observation so let’s record it as a proposition.

Example 6.2.9.

Suppose that W is a subspace of R4 having basis
w1=[1021],w2=[2134],
and that we wish to describe the orthogonal complement W.
If A is the matrix A=[w1w2] and x is in W, we have
ATx=[w1xw2x]=[00].
Describing vectors x that are orthogonal to both w1 and w2 is therefore equivalent to the more familiar task of describing the solution set ATx=0. To do so, we find the reduced row echelon form of AT and write the solution set parametrically as
x=x3[2110]+x4[1201].
Once again, the distributive property of dot products tells us that such a vector is also orthogonal to any linear combination of w1 and w2 so this solution set is, in fact, the orthogonal complement W. Indeed, we see that the vectors
v1=[2110],v2=[1201]
form a basis for W, which is a two-dimensional subspace of R4.
To place this example in a slightly more general context, note that w1 and w2, the columns of A, form a basis of W. Since Col(A), the column space of A is the subspace of linear combinations of the columns of A, we have W=Col(A).
This example also shows that the orthogonal complement W=Col(A) is described by the solution set of ATx=0. This solution set is what we have called Nul(AT), the null space of AT. In this way, we see the following proposition, which is visually represented in Figure 6.2.11.
Figure 6.2.11. The orthogonal complement of the column space of A is the null space of AT.

Subsection 6.2.3 Properties of the matrix transpose

The transpose is a simple algebraic operation performed on a matrix. The next activity explores some of its properties.

Activity 6.2.4.

In Sage, the transpose of a matrix A is given by A.T. Define the matrices
A=[103221],B=[341012],C=[103221320].
  1. Evaluate (A+B)T and AT+BT. What do you notice about the relationship between these two matrices?
  2. What happens if you transpose a matrix twice; that is, what is (AT)T?
  3. Find det(C) and det(CT). What do you notice about the relationship between these determinants?
    1. Find the product AC and its transpose (AC)T.
    2. Is it possible to compute the product ATCT? Explain why or why not.
    3. Find the product CTAT and compare it to (AC)T. What do you notice about the relationship between these two matrices?
  4. What is the transpose of the identity matrix I?
  5. If a square matrix D is invertible, explain why you can guarantee that DT is invertible and why (DT)1=(D1)T.
In spite of the fact that we are looking at some specific examples, this activity demonstrates the following general properties of the transpose, which may be verified with a little effort.

Properties of the transpose.

Here are some properties of the matrix transpose, expressed in terms of general matrices A, B, and C. We assume that C is a square matrix.
  • If A+B is defined, then (A+B)T=AT+BT.
  • (sA)T=sAT.
  • (AT)T=A.
  • det(C)=det(CT).
  • If AB is defined, then (AB)T=BTAT. Notice that the order of the multiplication is reversed.
  • If C is invertible, then (CT)1=(C1)T.
There is one final property we wish to record though we will wait until Section 7.4 to explain why it is true.
This proposition is important because it implies a relationship between the dimensions of a subspace and its orthogonal complement. For instance, if A is an m×n matrix, we saw in Section 3.5 that dimCol(A)=rank(A) and dimNul(A)=nrank(A).
Now suppose that W is an n-dimensional subspace of Rm with basis w1,w2,,wn. If we form the m×n matrix A=[w1w2wn], then Col(A)=W so that
rank(A)=dimCol(A)=dimW=n.
The transpose AT is an n×m matrix having rank(AT)=rank(A)=n. Since W=Nul(AT), we have
dimW=dimNul(AT)=mrank(AT)=mn=mdimW.
This explains the following proposition.

Example 6.2.14.

In Example 6.2.4, we constructed the orthogonal complement of a line in R3. The dimension of the orthogonal complement should be 31=2, which explains why we found the orthogonal complement to be a plane.

Example 6.2.15.

In Example 6.2.5, we looked at W, a 2-dimensional subspace of R5 and found its orthogonal complement W to be a 52=3-dimensional subspace of R5.

Activity 6.2.5.

  1. Suppose that W is a 5-dimensional subspace of R9 and that A is a matrix whose columns form a basis for W; that is, Col(A)=W.
    1. What is the shape of A?
    2. What is the rank of A?
    3. What is the shape of AT?
    4. What is the rank of AT?
    5. What is dimNul(AT)?
    6. What is dimW?
    7. How are the dimensions of W and W related?
  2. Suppose that W is a subspace of R4 having basis
    w1=[1021],w2=[1263].
    1. Find the dimensions dimW and dimW.
    2. Find a basis for W. It may be helpful to know that the Sage command A.right_kernel() produces a basis for Nul(A).
    3. Verify that each of the basis vectors you found for W are orthogonal to the basis vectors for W.

Subsection 6.2.4 Summary

This section introduced the matrix transpose, its connection to dot products, and its use in describing the orthogonal complement of a subspace.
  • The columns of the matrix A are the rows of the matrix transpose AT.
  • The components of the product ATx are the dot products of x with the columns of A.
  • The orthogonal complement of the column space of A equals the null space of AT; that is, Col(A)=Nul(AT).
  • If W is a subspace of Rp, then
    dimW+dimW=p.

Exercises 6.2.5 Exercises

1.

Suppose that W is a subspace of R4 with basis
w1=[2224],w2=[2355].
  1. What are the dimensions dimW and dimW?
  2. Find a basis for W.
  3. Verify that each of the basis vectors for W are orthogonal to w1 and w2.

2.

Consider the matrix A=[122134212].
  1. Find rank(A) and a basis for Col(A).
  2. Determine the dimension of Col(A) and find a basis for it.

3.

Suppose that W is the subspace of R4 defined as the solution set of the equation
x13x2+5x32x4=0.
  1. What are the dimensions dimW and dimW?
  2. Find a basis for W.
  3. Find a basis for W.
  4. In general, how can you easily find a basis for W when W is defined by
    Ax1+Bx2+Cx3+Dx4=0?

4.

Determine whether the following statements are true or false and explain your reasoning.
  1. If A=[211131], then x=[451] is in Col(A).
  2. If A is a 2×3 matrix and B is a 3×4 matrix, then (AB)T=ATBT is a 4×2 matrix.
  3. If the columns of A are v1, v2, and v3 and ATx=[201], then x is orthogonal to v2.
  4. If A is a 4×4 matrix with rank(A)=3, then Col(A) is a line in R4.
  5. If A is a 5×7 matrix with rank(A)=5, then rank(AT)=7.

5.

Apply properties of matrix operations to simplify the following expressions.
  1. AT(BAT)1
  2. (A+B)T(A+B)
  3. [A(A+B)T]T
  4. (A+2I)T

6.

A symmetric matrix A is one for which A=AT.
  1. Explain why a symmetric matrix must be square.
  2. If A and B are general matrices and D is a square diagonal matrix, which of the following matrices can you guarantee are symmetric?
    1. D
    2. BAB1
    3. AAT.
    4. BDBT

7.

If A is a square matrix, remember that the characteristic polynomial of A is det(AλI) and that the roots of the characteristic polynomial are the eigenvalues of A.
  1. Explain why A and AT have the same characteristic polynomial.
  2. Explain why A and AT have the same set of eigenvalues.
  3. Suppose that A is diagonalizable with diagonalization A=PDP1. Explain why AT is diagonalizable and find a diagonalization.

8.

This exercise introduces a version of the Pythagorean theorem that we’ll use later.
  1. Suppose that v and w are orthogonal to one another. Use the dot product to explain why
    |v+w|2=|v|2+|w|2.
  2. Suppose that W is a subspace of Rm and that z is a vector in Rm for which
    z=x+y,
    where x is in W and y is in W. Explain why
    |z|2=|x|2+|y|2,
    which is an expression of the Pythagorean theorem.

9.

In the next chapter, symmetric matrices---that is, matrices for which A=AT---play an important role. It turns out that eigenvectors of a symmetric matrix that are associated to different eigenvalues are orthogonal. We will explain this fact in this exercise.
  1. Viewing a vector as a matrix having one column, we may write xy=xTy. If A is a matrix, explain why x(Ay)=(ATx)y.
  2. We have seen that the matrix A=[1221] has eigenvectors v1=[11], with associated eigenvalue λ1=3, and v2=[11], with associated eigenvalue λ2=1. Verify that A is symmetric and that v1 and v2 are orthogonal.
  3. Suppose that A is a general symmetric matrix and that v1 is an eigenvector associated to eigenvalue λ1 and that v2 is an eigenvector associated to a different eigenvalue λ2. Beginning with v1(Av2), apply the identity from the first part of this exercise to explain why v1 and v2 are orthogonal.

10.

Given an m×n matrix A, the row space of A is the column space of AT; that is, Row(A)=Col(AT).
  1. Suppose that A is a 7×15 matrix. For what p is Row(A) a subspace of Rp?
  2. How can Proposition 6.2.10 help us describe Row(A)?
  3. Suppose that A=[122124151203]. Find bases for Row(A) and Row(A).
You have attempted 1 of 1 activities on this page.