Linear Algebra and Differential Equations

Lecture 20 - Orthogonal Matrices

`

Our Plan

• Rotations, Revisited
• Orthogonal Matrices and Isometries
• Example: Reflections

​

Rotations, Revisited

A rotation is a linear transformation that rotates a vector x counterclockwise by an angle φ (when φ < 0, the rotation is clockwise)

Rotations, Revisited

A rotation is a linear transformation that rotates a vector x counterclockwise by an angle φ (when φ < 0, the rotation is clockwise)

We denote its matrix by A_φ

Rotations, Revisited

Note that its determinant is

Rotations, Revisited

Note that its determinant is

so the matrix is invertible, with inverse

Clockwise Rotations

The clockwise rotation by an angle φ is

Clockwise Rotations

The clockwise rotation by an angle φ is

where we have used the fact that cos is an even function, while sin is an odd function

Clockwise Rotations

The clockwise rotation by an angle φ is

where we have used the fact that cos is an even function, while sin is an odd function

Remark

To invert a counterclockwise rotation by φ, one simply has to rotate clockwise by the same angle φ

Remark

To invert a counterclockwise rotation by φ, one simply has to rotate clockwise by the same angle φ

or

Remark

To invert a counterclockwise rotation by φ, one simply has to rotate clockwise by the same angle φ

or

That is, rotating counterclockwise and then clockwise returns every vector to itself

Remark

Comparing the inverse with the original matrix, here is another interesting property

Remark

Comparing the inverse with the original matrix, here is another interesting property

or

Remark

Comparing the inverse with the original matrix, here is another interesting property

or

More Algebraic Properties

The null space of A_φ is trivial

More Algebraic Properties

The null space of A_φ is trivial

Because the only vector that gets rotated to 0 is 0 itself

More Algebraic Properties

The null space of A_φ is trivial

Because the only vector that gets rotated to 0 is 0 itself

Therefore

More Algebraic Properties

Since the rank is 2, the range is all of ℝ²

More Algebraic Properties

Since the rank is 2, the range is all of ℝ²

Indeed, any vector y in ℝ² can be interpreted as the rotated version (counterclockwise by φ) of another vector x. In other words, the following is always consistent

Orthogonal Matrices

Matrices with this property have a special name (this holds for any n)

Orthogonal Matrices

Matrices with this property have a special name (this holds for any n)

Definition: An (n × n) matrix A for which

is called an orthogonal matrix. In other words, A is invertible with

Isometries

Definition: A linear transformation

is called an isometry, or an isometric transformation, if

Isometries

Definition: A linear transformation

is called an isometry, or an isometric transformation, if

In other words, an isometry preserves the length of vectors: T(x) has the same length as x, for all x ∈ ℝⁿ

Isometries

If A is the (n × n) matrix associated with an isometry T, i.e.

Isometries

If A is the (n × n) matrix associated with an isometry T, i.e.

then we have

Isometries

If A is the (n × n) matrix associated with an isometry T, i.e.

then we have

Note that rotations in ℝ² are isometries! Rotating a vector, counterclockwise or clockwise, by an angle φ does not change its magnitude

A Natural Question

So rotations are isometries, and rotation matrices are orthogonal. A natural question is the following one:

A Natural Question

So rotations are isometries, and rotation matrices are orthogonal. A natural question is the following one:

Are the matrices of isometries all orthogonal?

A Natural Question

So rotations are isometries, and rotation matrices are orthogonal. A natural question is the following one:

Are the matrices of isometries all orthogonal?

Vice versa, do all orthogonal matrices define isometric transformations?

A Natural Question

So rotations are isometries, and rotation matrices are orthogonal. A natural question is the following one:

Are the matrices of isometries all orthogonal?

Vice versa, do all orthogonal matrices define isometric transformations?

Orthogonal Matrices and Isometries

Theorem: Let A be an (n × n) matrix, written by its columns

The following three facts are equivalent:

Orthogonal Matrices and Isometries

Theorem: Let A be an (n × n) matrix, written by its columns

The following three facts are equivalent:

Orthogonal Matrices and Isometries

Theorem: Let A be an (n × n) matrix, written by its columns

The following three facts are equivalent:

Orthogonal Matrices and Isometries

Theorem: Let A be an (n × n) matrix, written by its columns

The following three facts are equivalent:

Orthogonal Matrices and Isometries

In words: (I) says A is an orthogonal matrix, (II) says the linear transformation y = Ax is an isometry, and (III) says the columns are orthonormal, i.e.

Orthogonal Matrices and Isometries

In words: (I) says A is an orthogonal matrix, (II) says the linear transformation y = Ax is an isometry, and (III) says the columns are orthonormal, i.e.

Remark: By last lecture, an orthonormal set of n vectors in ℝⁿ is linearly independent. Therefore, they form an orthonormal basis of ℝⁿ

Example: Rotations

Before we proceed, note that all three facts are true for the (2 × 2) rotation matrix

Example: Rotations

Before we proceed, note that all three facts are true for the (2 × 2) rotation matrix

In particular, it is easy to verify that the columns are indeed orthonormal

Proof Strategy

Instead of employing a scheme that would require four proofs, we will implement a “triangular” scheme. This only requires three proofs

Proof Strategy

Instead of employing a scheme that would require four proofs, we will implement a “triangular” scheme. This only requires three proofs

(I) ⟹ (II) and (III) ⟹ (I) are relatively simple

Proof Strategy

Instead of employing a scheme that would require four proofs, we will implement a “triangular” scheme. This only requires three proofs

(I) ⟹ (II) and (III) ⟹ (I) are relatively simple

(II) ⟹ (III) is a bit more complex

Proof Strategy

Instead of employing a scheme that would require four proofs, we will implement a “triangular” scheme. This only requires three proofs

(I) ⟹ (II) and (III) ⟹ (I) are relatively simple

(II) ⟹ (III) is a bit more complex

A Lemma

Lemma: Let A be an (n × n) matrix written by its columns, and consider the standard basis

Then it is the case that

A Lemma

Lemma: Let A be an (n × n) matrix written by its columns, and consider the standard basis

Then it is the case that

Proof of the Lemma

Note that for a generic vector

Proof of the Lemma

Note that for a generic vector

we have

Proof of the Lemma

Proof of the Lemma

Now, the standard unit vector e_i has

Proof of the Lemma

Now, the standard unit vector e_i has

Therefore

Proof: (I) ⟹ (II)

This is the easy part. Assume

Proof: (I) ⟹ (II)

This is the easy part. Assume

Then we have, for any x ∈ ℝⁿ:

Proof: (I) ⟹ (II)

This is the easy part. Assume

Then we have, for any x ∈ ℝⁿ:

so

Proof: (II) ⟹ (III)

This is the trickiest. Now assume that

Proof: (II) ⟹ (III)

This is the trickiest. Now assume that

Then, in particular, this must be true for the standard unit vectors

Proof: (II) ⟹ (III)

This is the trickiest. Now assume that

Then, in particular, this must be true for the standard unit vectors

However, note that

Proof: (II) ⟹ (III)

Inserting these into ‖Ae_i‖ = ‖e_i‖ yields

Proof: (II) ⟹ (III)

Inserting these into ‖Ae_i‖ = ‖e_i‖ yields

Proof: (II) ⟹ (III)

Inserting these into ‖Ae_i‖ = ‖e_i‖ yields

So every column of A is a unit vector. Next, we show the columns are pairwise orthogonal

Proof: (II) ⟹ (III)

Now take two arbitrary indices i, j, with i ≠ j, and consider

Proof: (II) ⟹ (III)

Now take two arbitrary indices i, j, with i ≠ j, and consider

which has a 1 in the i-th position, a 1 in the j-th position, and zeros elsewhere. Note that

Proof: (II) ⟹ (III)

For the same vector x, let’s now compute

Proof: (II) ⟹ (III)

For the same vector x, let’s now compute

Proof: (II) ⟹ (III)

For the same vector x, let’s now compute

Proof: (II) ⟹ (III)

Because of the assumption that we are making, it must be the case that

Proof: (II) ⟹ (III)

Because of the assumption that we are making, it must be the case that

i.e.

Proof: (II) ⟹ (III)

Because of the assumption that we are making, it must be the case that

i.e.

Proof: (III) ⟹ (I)

Finally, assume that

Proof: (III) ⟹ (I)

Finally, assume that

and write A by its columns

Proof: (III) ⟹ (I)

Proof: (III) ⟹ (I)

Therefore, we have used the orthonormality of the set, i.e.

Proof: (III) ⟹ (I)

where in (∗∗∗∗) we have used the orthonormality of the set, i.e.

In conclusion, we have that

On the Use of the Word “Orthogonal”

The word “orthogonal” has different meanings, depending on what it refers to

On the Use of the Word “Orthogonal”

The word “orthogonal” has different meanings, depending on what it refers to

Two orthogonal vectors u and v are vectors whose dot product is zero

On the Use of the Word “Orthogonal”

The word “orthogonal” has different meanings, depending on what it refers to

Two orthogonal vectors u and v are vectors whose dot product is zero

An orthogonal set of vectors is a set of vectors that are pairwise orthogonal

On the Use of the Word “Orthogonal”

On the other hand, an orthogonal (n × n) matrix A is a matrix whose inverse is given by its own transpose. By the theorem,

On the Use of the Word “Orthogonal”

On the other hand, an orthogonal (n × n) matrix A is a matrix whose inverse is given by its own transpose. By the theorem,

i.e., its columns are orthogonal and of unit length

On the Use of the Word “Orthogonal”

On the other hand, an orthogonal (n × n) matrix A is a matrix whose inverse is given by its own transpose. By the theorem,

i.e., its columns are orthogonal and of unit length

On Orthonormal Bases

If the following is an orthonormal basis of ℝⁿ

On Orthonormal Bases

If the following is an orthonormal basis of ℝⁿ

then the matrix of the basis is an orthogonal matrix

On Orthonormal Bases

So the change of coordinates matrix (from the canonical basis α of ℝⁿ to the basis β)

On Orthonormal Bases

So the change of coordinates matrix (from the canonical basis α of ℝⁿ to the basis β)

may simply be computed as

On Orthonormal Bases

So the change of coordinates matrix (from the canonical basis α of ℝⁿ to the basis β)

may simply be computed as

Example: Reflection

Consider the transformation T where, for a generic x ∈ ℝ², T(x) is the mirror reflection of x with respect to the line x₁ = x₂

Example: Reflection

Note that if x has coordinates a and b, then T(x) has coordinates b and a

Example: Reflection

Note that if x has coordinates a and b, then T(x) has coordinates b and a

Example: Reflection

Therefore

Example: Reflection

Therefore

which we rewrite as

Example: Reflection

Therefore

which we rewrite as

In other words, the transformation T is linear, with matrix the matrix

Properties of the Reflection Matrix

This matrix is very special: it has a lot of properties

(a)

Properties of the Reflection Matrix

This matrix is very special: it has a lot of properties

(a)

So A is its own inverse! In symbols:

Properties of the Reflection Matrix

This matrix is very special: it has a lot of properties

(a)

So A is its own inverse! In symbols:

Matrices with this property are called involutory (and the corresponding transformations T involutions)

Properties of the Reflection Matrix

Geometrically, this makes sense: to recover x from T(x) = Ax you have to mirror-reflect again with respect to x₁ = x₂, i.e. multiply by A again

Properties of the Reflection Matrix

(b) A is symmetric:

(a)

Properties of the Reflection Matrix

(c) Combining (a) and (b) yields

(b) A is symmetric:

(a)

Properties of the Reflection Matrix

In agreement with the theorem, it should be fairly obvious that the columns of A

Properties of the Reflection Matrix

In agreement with the theorem, it should be fairly obvious that the columns of A

form an orthonormal set, and that the mirror reflection preserves length

Range and Null Space

Finally, because A is nonsingular, we have that

Range and Null Space

Finally, because A is nonsingular, we have that

Every vector y ∈ ℝ² can be obtained as the reflection of some other vector x ∈ ℝ², i.e. the following is always consistent

Range and Null Space

Finally, because A is nonsingular, we have that

Every vector y ∈ ℝ² can be obtained as the reflection of some other vector x ∈ ℝ², i.e. the following is always consistent

Also, the only vector that gets reflected to 0 is the vector 0 itself