5. Orthogonal matrices PDF R × has orthonormal columns if

6.3 Orthogonal and orthonormal vectors

6.3 Orthogonal and orthonormal vectors. Definition. We say that 2 vectors are orthogonal if they are perpendicular to each other.

5. Orthogonal matrices

R × has orthonormal columns if its Gram matrix is the identity matrix: a square real matrix with orthonormal columns is called orthogonal.

21. Orthonormal Bases

In addition to being orthogonal each vector has unit length. Suppose T = {u1

Orthogonal but not Orthonormal

https://empslocal.ex.ac.uk/people/staff/reverson/uploads/Site/procrustes.pdf

Lecture 4 Orthonormal sets of vectors and QR factorization

slang: we say 'u1

Orthogonal and orthonormal sets

24-Feb-2015 Note that if S is orthonormal then o ? S

Math 115A - Week 9 Textbook sections: 6.1-6.2 Topics covered

Orthonormal bases. • Gram-Schmidt orthogonalization. • Orthogonal complements. *****. Orthogonality. • From your lower-division vector calculus you know

Orthogonality

slang: we say 'u1;:::;uk are orthonormal vectors' but orthonormality (like I (you'd think such matrices would be called orthonormal not orthogonal).

Orthonormal Bases in Hilbert Space APPM 5440 Fall 2017 Applied

02-Dec-2017 Let (ek) be an orthonormal sequence in an inner product space X. Let x ? X. The quantities. ?ekx? are called the Fourier coefficients of x ...

Orthonormal Sets • Bessel Inequality • Total Orthonormal Sequences

EL3370 Orthogonal Expansions - 5. BESSEL INEQUALITY. Theorem (Bessel Inequality). If !! {e n. } is an orthonormal sequence in an inner product space V

L. VandenbergheECE133A (Fall 2022)

5. Orthogonal matrices

matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5.1

Orthonormal vectors

a collection of real<-vectors01,02, ...,0=isorthonormalif the vectors have unit norm:k08k=1 they are mutually orthogonal:0)809=0if8<9

Example

666640

0 13

77775-1p2

666641

1 03

77775-1p2

666641

1 03 77775

Orthogonal matrices5.2

Matrix with orthonormal columns

2R<=has orthonormal columns if its Gram matrix is the identity matrix:

)=01020=)01020= 2

666666640

)1010)1020)10= 0 0 )=010)=020)=0=3

77777775

66666641 00

0 10

0 013

7777775

there is no standard short name for "matrix with orthonormal columns"

Orthogonal matrices5.3

Matrix-vector product

if2R<=has orthonormal columns, then the linear function5¹Gº=G preserves inner products:

¹Gº)¹Hº=G))H=G)H

preserves norms: kGk=

¹Gº)¹Gº

12=¹G)Gº12=kGk

preserves distances:kGHk=kGHk preserves angles: \¹G- Hº=arccos¹Gº)¹HºkGkkHk =arccosG)HkGkkHk =\¹G- Hº

Orthogonal matrices5.4

Left-invertibility

if2R<=has orthonormal columns, then is left-invertible with left inverse): by definition has linearly independent columns (from page 4.23 or page5.2 ):

G=0=))G=G=0

is tall or square:<=(see page 4.12)

Orthogonal matrices5.5

Outline

matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns

Orthogonal matrix

asquarereal matrix with orthonormal columns is calledorthogonal Nonsingularity(from equivalences on page 4.13): ifis orthogonal, then is invertible, with inverse): is square )is also an orthogonal matrix rows ofare orthonormal (have norm one and are mutually orthogonal) Note:if2R<=has orthonormal columns and< ¡ =, then)<

Orthogonal matrices5.6

Permutation matrix

letc=¹c1- c2--c=ºbe a permutation (reordering) of¹1-2--=º we associate withcthe==permutation matrix

8c8=1- 89=0if9 Gis a permutation of the elements ofG:G=¹Gc1-Gc2--Gc=º has exactly one element equal to 1 in each row and each column
Orthogonality:permutation matrices are orthogonal
)=becausehas one element equal to one in each row and column
¹)º89==X
:=1 :8: 9=18=9
0otherwise
)=1is the inverse permutation matrix
Orthogonal matrices5.7

Example
permutation onf1-2-3-4g
¹c1- c2- c3- c4º=¹2-4-1-3º
corresponding permutation matrix and its inverse =2
66666640 1 0 0

0 0 0 1

1 0 0 0

0 0 1 03

7777775-
1=)=2
66666640 0 1 0

1 0 0 0

0 0 0 1

0 1 0 03

7777775
)is permutation matrix associated with the permutation
Orthogonal matrices5.8

Plane rotation

Rotation in a plane
=cos\sin\ sin\cos\??? \Rotation in a coordinate plane inR=:for example, =266664cos\0sin\ 0 1 0 sin\0 cos\3 77775
describes a rotation in the¹G1-G3ºplane inR3
Orthogonal matrices5.9

Reflector

Reflector:a matrix of the form
=200) with0a unit-norm vector (k0k=1)
Properties
a reflector matrix is symmetric a reflector matrix is orthogonal
Orthogonal matrices5.10

Geometrical interpretation of reflector
?=??=¹?2???º?? line through?and origin? ?=¹????º?0=fDj0)D=0gis the (hyper-)plane of vectors orthogonal to0 ifk0k=1, the projection ofGonis given by
H=G ¹0)Gº0=G0¹0)Gº=¹00)ºG
(see next page) reflection ofGthrough the hyperplane is given by product with reflector:
I=H¸ ¹HGº=¹200)ºG

Orthogonal matrices5.11

Exercise
supposek0k=1; show that the projection ofGon=fDj0)D=0gis
H=G ¹0)Gº0
we verify thatH2: 0 )H=0)¹G0¹0)Gºº=0)G ¹0)0º¹0)Gº=0)G0)G=0 now consider anyI2withI¡kGHk2

Orthogonal matrices5.12

Product of orthogonal matrices

if1, ...,:are orthogonal matrices and of equal size, then the product =12: is orthogonal: )=¹12:º)¹12:º =):)2)112:

Orthogonal matrices5.13

Linear equation with orthogonal matrix

linear equation with orthogonal coefficient matrixof size== G=1 solution is

G=11=)1

can be computed in2=2flops by matrix-vector multiplication cost is less than order=2ifhas special properties; for example, permutation matrix:0flops reflector (given0): order=flops plane rotation: order1flops

Orthogonal matrices5.14

Outline

matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns

Tall matrix with orthonormal columns

suppose2R<=is tall (< ¡ =) and has orthonormal columns )is a left inverse of: has no right inverse; in particular on the next pages, we give a geometric interpretation to the matrix)

Orthogonal matrices5.15

Range thespanof a collection of vectors is the set of all their linear combinations: span¹01-02--0=º=fG101¸G202¸ ¸G=0=jG2R=g therangeof a matrix2R<=is the span of its column vectors: range¹º=fGjG2R=g

Example

range¹2666641 0 1 2 013

77775º=8

>:2

66664G

1 G

1¸2G2

G23

77775jG1-G22R9

Orthogonal matrices5.16

Projection on range of matrix with orthonormal columns suppose2R<=has orthonormal columns01--0=; we show that the vector )1 is the orthogonal projection of an<-vector1onrange¹ºrange(?)? ??on next page we show thatˆG=)1satisfieskˆG1kkG1kfor allG<ˆG this extends the result on page 2.12 (where=¹1k0kº0)

1)1=¹)º1is theresidualof1after the projection

Orthogonal matrices5.17

Proof the squared distance of1to an arbitrary pointGinrange¹ºis kG1k2=k¹GˆGº ¸ˆG1k2(whereˆG=)1) =k¹GˆGºk2¸ kˆG1k2¸2¹GˆGº))¹ˆG1º =k¹GˆGºk2¸ kˆG1k2 =kGˆGk2¸ kˆG1k2 kˆG1k2 with equality only ifG=ˆG line 3 follows because)¹ˆG1º=ˆG)1=0 line 4 follows from)=

Orthogonal matrices5.18

Orthogonal decomposition

the vector1is decomposed as a sum1=I¸Hwith

I2range¹º- H?range¹ºrange(?)?

?=?????=?-????such a decomposition exists and is unique for every1:

1=G¸H- )H=0()G=)1- H=1)1

ifhas orthonormal columns

Orthogonal matrices5.19

Outline

matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns

Gram matrix

2C<=has orthonormal columns if its Gram matrix is the identity matrix:

=01020=01020= 2

666666640

1010102010=

0 0 =010=020=0=3

77777775

66666641 00

0 10

0 013

7777775

columns have unit norm:k08k2=0808=1 columns are mutually orthogonal:0809=0for8<9

Orthogonal matrices5.20

Unitary matrix

asquarecomplex matrix with orthonormal columns is calledunitary

Inverse

is square a unitary matrix is nonsingular with inverse ifis unitary, thenis unitary

Orthogonal matrices5.21

Discrete Fourier transform matrix

recall definition from page 3.37 (withl=42cj=andj=p1) ,=2

6666666641 1 11

1l1l2l¹=1º

1l2l4l2¹=1º

777777775

the matrix¹1p=º,is unitary (proof on next page): 1= ,,=1= inverse of,is,1=¹1=º, inverse discrete Fourier transform of=-vectorGis,1G=¹1=º,G

Orthogonal matrices5.22

Gram matrix of DFT matrix

we show that,,== conjugate transpose of,is =2

6666666641 1 11

1l l2l=1

1l2l4l2¹=1º

1l=1l2¹=1ºl¹=1º¹=1º3

777777775

8- 9element of Gram matrix is

¹,,º89=1¸l89¸l2¹89º¸ ¸l¹=1º¹89º

891=0if8<9

(last step follows froml==1)

Orthogonal matrices5.23

quotesdbs_dbs19.pdfusesText_25

[PDF] 5. Orthogonal matrices R × has orthonormal columns if

L. VandenbergheECE133A (Fall 2022)

5. Orthogonal matrices

Orthonormal vectors

Example

666640

77775-1p2

666641

77775-1p2

666641

Orthogonal matrices5.2

Matrix with orthonormal columns

2R<=has orthonormal columns if its Gram matrix is the identity matrix:

666666640

77777775

66666641 00

0 10

7777775

Orthogonal matrices5.3

Matrix-vector product

¹Gº)¹Hº=G))H=G)H

¹Gº)¹Gº

12=¹G)Gº12=kGk

Orthogonal matrices5.4

Left-invertibility

G=0=))G=G=0

Orthogonal matrices5.5

Outline

Orthogonal matrix

Orthogonal matrix

Orthogonal matrices5.6

Permutation matrix

Orthogonality:permutation matrices are orthogonal

¹)º89==X

0otherwise

Orthogonal matrices5.7

Example

¹c1- c2- c3- c4º=¹2-4-1-3º

66666640 1 0 0

0 0 0 1

1 0 0 0

0 0 1 03

7777775-

66666640 0 1 0

1 0 0 0

0 0 0 1

0 1 0 03

7777775

Orthogonal matrices5.8

Plane rotation

Rotation in a plane

Orthogonal matrices5.9

Reflector

Reflector:a matrix of the form

Properties

Orthogonal matrices5.10

Geometrical interpretation of reflector

H=G ¹0)Gº0=G0¹0)Gº=¹00)ºG

I=H¸ ¹HGº=¹200)ºG

Orthogonal matrices5.11

Exercise

H=G ¹0)Gº0

Orthogonal matrices5.12

Product of orthogonal matrices

Orthogonal matrices5.13

Linear equation with orthogonal matrix

G=11=)1

Orthogonal matrices5.14

Outline

Tall matrix with orthonormal columns

Orthogonal matrices5.15

Example

77775º=8

66664G

1¸2G2

77775jG1-G22R9

Orthogonal matrices5.16

1)1=¹)º1is theresidualof1after the projection

Orthogonal matrices5.17

Orthogonal matrices5.18

0 10

12=¹G)Gº12=kGk

0 10