1 of 406

LINEAR ALGEBRA

IV SEMESTER (ECE)

Linear Algebra and Its Applications by Gilbert Strang

1

2 of 406

BACKGROUND - VECTORS

  • A vector is a directed line segment that represents displacement from one point A to another B.
  • Examples: Force, velocity, acceleration, etc.

2

Here, A is called tail or initial point and B is called head or terminal point

3 of 406

BACKGROUND - VECTORS

  • A point can also be represented as a vector which is referred to as position vector.
    • Position vectors have their tails at the origin

​

3

 

4 of 406

BACKGROUND - VECTORS

  • Two vectors are equal if they have same length and same direction
    • The word vector means “carry to”. The vector [3,2] can be interpreted as starting from origin O, travel 3 units to the right and 2 units up, finishing at P.

4

 

5 of 406

BACKGROUND - VECTORS

  • If we want one vector u to follow another vector v, it leads to vector addition u + v
  • Example: u=[1,2] and v=[2,2], then u + v = [3,4]

5

6 of 406

BACKGROUND - VECTORS

  •  

6

 

7 of 406

BACKGROUND - VECTORS

  •  

7

8 of 406

BACKGROUND - VECTORS

  •  

8

9 of 406

BACKGROUND - VECTORS

  •  

9

 

10 of 406

BACKGROUND - VECTORS

  •  

10

11 of 406

BACKGROUND - VECTORS

  •  

11

12 of 406

Matrices, Gauss Elimination and Vector Spaces

UNIT 1

12

13 of 406

Introduction – Linear System

  •  

13

14 of 406

Introduction – Linear System

  •  

14

15 of 406

Introduction – Linear System

  •  

15

16 of 406

Introduction – Linear System

  •  

16

17 of 406

Introduction – Linear System

  •  

17

18 of 406

Introduction – Linear System

  •  

18

19 of 406

Introduction – Linear System

  •  

19

20 of 406

Introduction – Linear System

  • Scientific problems can run into 1000s of variables
  • Elimination method is preferred over ratio of determinants due to the computation complexity
  • Geometric representation and Matrix representation play an important role in understanding a system of linear equations.
    • Former helps analysis and latter helps solving

20

21 of 406

Geometric representation

  •  

21

22 of 406

Geometric representation – 2D

  •  

22

The only solution to both equations is the intersection point (2,3) given by elimination

23 of 406

Geometric representation – 2D

  •  

23

24 of 406

Geometric representation – 2D

  • Column picture:

24

 

25 of 406

Geometric representation – 3D

  •  

25

26 of 406

Geometric representation – 3D

26

27 of 406

Geometric representation – 3D

  •  

27

28 of 406

Geometric representation – 3D

  •  

28

29 of 406

Geometric representation – 3D

  •  

29

30 of 406

Geometric representation – 3D

30

31 of 406

Geometric representation – 3D

  •  

31

32 of 406

Geometric representation – 3D

  •  

32

33 of 406

 

  •  

33

34 of 406

 

  •  

34

35 of 406

Singular case

  • When there is no unique solution for a system of linear equations, we refer to it as singular case
  • In other words, the system has no solution or infinite solutions!
  • Consider a 3×3 system of linear equations. The singular case occurs when:
    • Two or more planes are parallel
    • The intersection of any pair of planes results in a line. However, the lines are parallel.

35

36 of 406

Singular case

​

​

​

​

​

  • We refer to the below system as inconsistent

36

LHS of third equation is sum of first and second equation

37 of 406

Singular case

  • Suppose we changed the right side of 3rd equation to 7.
  • Then the 3rd equation becomes parallel to the line formed by the intersection of the first two equations

37

7

38 of 406

Singular case

  •  

38

As column 1 is a linear combination of the other two columns, they are coplanar

39 of 406

Singular case

  • Column picture (contd.)

39

40 of 406

Singular case

  • Summary
    • If the row picture breaks down, so does the column picture.
    • As the rows were linear combinations of one another, so would the columns be linear combinations of one another
    • If the n planes have no point in common, or infinitely many points, then the n columns lie in the same plane
    • We need a good algorithm and notation to detect solution or singularity

​

40

41 of 406

Singular case

  • Problem 4:
    • Explain why the system below is singular by finding a combination of the three equations that adds up to 0 = 1.

u + v + w = 2

u + 2v + 3w = 1

v + 2w = 0

    • What value should replace the last zero on the right side to allow the equations to have solutions—and what is one of the solutions?

41

42 of 406

Gauss elimination

  •  

42

43 of 406

Gauss elimination

  •  

43

44 of 406

Gauss elimination

  •  

44

45 of 406

Gauss elimination

  •  

45

46 of 406

Gauss elimination

  •  

46

47 of 406

Gauss elimination - Breakdowns

  • During forward eliminations, a pivot element can become zero
    • It does not mean that the algorithm terminated.
    • There is no way to ascertain that the coefficient matrix is singular or not.
    • In such cases we will try row exchanges to overcome the situation.
    • However, if there is no possibility of recovering the situation, then we can conclude that the singular case occurred.

47

48 of 406

Gauss elimination - Breakdowns

  • Examples:
    • Nonsingular (cured by exchanging equations 2 and 3)

​

​

    • The above system is now triangular so we can apply back substitution and solve it

​

48

49 of 406

Gauss elimination - Breakdowns

  •  

49

50 of 406

Gauss elimination - Breakdowns

  • Problem 5:
    • Solve the following systems of equations using Gaussian elimination

2x + y + 3z = 1 , 2x + 6y + 8z = 3 , 6x + 8y + 18 z = 5

  • Problem 6:
    • Solve the following systems of equations using Gaussian elimination

3x + y – 6z = –10 , 2x + y – 5z = –8 , 6x – 3y + 3z = 0

50

51 of 406

Gauss elimination - Breakdowns

  • Problem 7:
    • Solve the following systems of equations using Gaussian elimination

x + z = 1, x + y + z = 2, x – y + z = 1.

    • What if the right-hand side is ( 1, 2, 0 ) ?

​

51

52 of 406

Gauss elimination - Breakdowns

  • Problem 8:
    • Determine the values of a and b for which the system of equations x + y + a z = 2b , x + 3y + (2 + 2a) z = 7b , 3x + y + ( 3 + 3a ) z = 11b will have
      • Unique nontrivial solution
      • Trivial solution
      • No solution
      • Infinity of solutions.

52

53 of 406

Gauss elimination - Breakdowns

  •  

53

54 of 406

Elementary matrices

  •  

54

55 of 406

Elementary matrices

  •  

55

56 of 406

Elementary matrices

  •  

56

57 of 406

Elementary matrices

  •  

57

58 of 406

Elementary matrices

  •  

58

59 of 406

Elementary matrices

  •  

59

60 of 406

LU Decomposition

  • Matrix A can be factorized into upper triangular matrix U and lower triangular matrix L
  • Triangular matrices are easy to work with (e.g., finding solution, determinants and inverses)
  • Computers exploit the LU decomposition of A
  • Matrix A is converted into U using forward eliminations (i.e., multiplication by a series of elementary matrices)
  • The matrix U is converted into matrix A in one step by multiplying it with matrix L

60

61 of 406

LU Decomposition

  •  

61

62 of 406

LU Decomposition

  •  

62

63 of 406

LU Decomposition

  •  

63

64 of 406

LU Decomposition

  •  

64

65 of 406

LU Decomposition

  •  

65

66 of 406

LU Decomposition

  •  

66

 

67 of 406

LU Decomposition

  •  

67

 

68 of 406

LU Decomposition

  •  

68

69 of 406

LU Decomposition

  •  

69

70 of 406

LU Decomposition

  •  

70

71 of 406

LU Decomposition

  •  

71

72 of 406

LU Decomposition

  •  

72

73 of 406

LU Decomposition

  •  

73

74 of 406

LU Decomposition

  •  

74

75 of 406

LU Decomposition

  •  

75

76 of 406

LU Decomposition

  • Sometimes, we call this as LDU decomposition
    • Note that the U in LDU decomposition is not the same as the U in LU decomposition
  • Example:

​

76

77 of 406

Cholesky Decomposition

  •  

77

78 of 406

Cholesky Decomposition

  •  

78

79 of 406

Cholesky Decomposition

  •  

79

80 of 406

Cholesky Decomposition

  •  

80

81 of 406

Cholesky Decomposition

  • From PESU Academy (contd.)

81

82 of 406

Cholesky Decomposition

  • From PESU Academy (contd.)

82

83 of 406

Permutation matrices

  •  

83

84 of 406

Permutation matrices

  •  

84

85 of 406

Permutation matrices

  •  

85

86 of 406

Permutation matrices

  •  

86

87 of 406

Permutation matrices

  •  

87

88 of 406

Permutation matrices

  •  

88

89 of 406

Permutation matrices

  •  

89

90 of 406

Permutation matrices

  • Problem 12:
    • Find 𝑃𝐴=𝐿𝐷𝑈 factorization for following matrices

​

i)

​

90

91 of 406

Numerical problems

  •  

91

92 of 406

Inverse and transpose

  •  

92

93 of 406

Inverse and transpose

  •  

93

94 of 406

Inverse and transpose

  •  

94

95 of 406

Inverse and transpose

  •  

95

96 of 406

Inverse and transpose

  •  

96

97 of 406

Inverse and transpose

  •  

97

98 of 406

Inverse and transpose

  •  

98

99 of 406

Numerical problems

  •  

99

100 of 406

Vector spaces and subspaces

  • A vector space is a non-empty set of vectors on which are defined two operations, namely, addition and multiplication by scalars, subject to 10 axioms (or rules) listed below.
    • The axioms must hold for all vectors u, v, and w in V and for all scalars c and d.
  • The sum of u and v, denoted by u + v, is in V
  • u + v = v + u
  • (u + v) + w = u + (v + w)

100

101 of 406

Vector spaces and subspaces

  • There is a zero vector in V such that u + 0 = u
  • For each u in V , there is a vector –u in V such that u + (– u) = 0.
  • The scalar multiple of u by c, denoted by cu, is in V.
  • c(u + v) = cu + cv.
  • (c + d)u = cu + du.
  • c(du) = (cd)u.
  • 1u = u

101

102 of 406

Vector spaces and subspaces

  •  

102

103 of 406

Vector spaces and subspaces

  •  

103

104 of 406

Vector spaces and subspaces

  • A subspace of a vector space V is a subset H of V that has three properties:
    • The zero vector of V is in H
    • H is closed under vector addition. That is, for each u and v in H, the sum u + v is in H.
    • H is closed under multiplication by scalars. That is, for each u in H and each scalar c, the vector cu is in H.

104

105 of 406

Vector spaces and subspaces

  •  

105

106 of 406

Vector spaces and subspaces

  •  

106

107 of 406

Vector spaces and subspaces

  •  

107

108 of 406

Vector spaces and subspaces

  •  

108

109 of 406

Four Fundamental Subspaces & Linear Transformations

UNIT 2

109

110 of 406

 

  •  

110

111 of 406

 

  •  

111

112 of 406

 

  •  

112

113 of 406

 

  •  

113

114 of 406

 

  • If a matrix in echelon form satisfies the following additional conditions, then it is in reduced echelon form (or reduced row echelon form):
    • The leading entry in each nonzero row is 1.
    • Each leading 1 is the only nonzero entry in its column.

114

115 of 406

 

  •  

115

116 of 406

 

  •  

116

117 of 406

 

  •  

117

118 of 406

 

  •  

118

119 of 406

 

  •  

119

120 of 406

 

  •  

120

121 of 406

 

  •  

121

122 of 406

 

  •  

122

u

v

123 of 406

 

    • The dimension of the nullspace is same as number of free variables and independent vectors
    • The general solution is a linear combination of the independent vectors u and v
    • The vectors u and v span the nullspace of A
    • If Ax = 0 has more unknowns than equations (n > m), it has at least one special solution: There are more solutions than the trivial x = 0.

123

124 of 406

 

  •  

124

125 of 406

 

  •  

125

126 of 406

 

  •  

126

127 of 406

Solutions and implications

  •  

127

128 of 406

Solutions and implications

  •  

128

129 of 406

Solutions and implications

  •  

129

130 of 406

Solutions and implications

  •  

130

131 of 406

Solutions and implications

  •  

131

132 of 406

Solutions and implications

  •  

132

133 of 406

Solutions and implications

  •  

133

134 of 406

Solutions and implications

  •  

134

135 of 406

Linear independence of vectors

  •  

135

136 of 406

Linear independence of vectors

  •  

136

137 of 406

Linear independence of vectors

  •  

137

138 of 406

Linear independence of vectors

  •  

138

139 of 406

Linear independence of vectors

  •  

139

140 of 406

Linear independence of vectors

  •  

140

141 of 406

Linear independence of vectors

  •  

141

142 of 406

Linear independence of vectors

  •  

142

143 of 406

Linear independence of vectors

  •  

143

144 of 406

Linear independence of vectors

  •  

144

145 of 406

Bases and Dimension

  •  

145

146 of 406

Bases and Dimension

  •  

146

147 of 406

Bases and Dimension

  •  

147

148 of 406

Bases and Dimension

  •  

148

149 of 406

Bases and Dimension

  • Any linearly independent set in V can be extended to a basis, by adding more vectors if necessary
    • The point is that a basis is a maximal independent set. It cannot be made larger without losing independence.
    • A basis is also a minimal spanning set. It cannot be made smaller and still span the space.
    • The dimension of the column space equals the rank of the matrix

149

150 of 406

Four fundamental subspaces

  •  

150

151 of 406

Four fundamental subspaces

  •  

151

152 of 406

 

  •  

152

153 of 406

 

  •  

153

154 of 406

 

  •  

154

155 of 406

 

  •  

155

156 of 406

 

  •  

156

157 of 406

 

    • Theorem: row rank = column rank
    • The dimension of the column space C(A) equals the rank r, which also equals the dimension of the row space of A.
    • The number of independent columns equals the number of independent rows.
    • A basis for C(A) is formed by the r columns of A that correspond, in U, to the columns containing pivots.

157

158 of 406

 

  •  

158

 

 

159 of 406

 

  •  

159

160 of 406

 

  •  

160

161 of 406

 

  •  

161

162 of 406

 

  •  

162

163 of 406

 

  •  

163

164 of 406

Numerical problems

  •  

164

165 of 406

Existence of inverses

  •  

165

166 of 406

Existence of inverses

  •  

166

167 of 406

Existence of inverses

  •  

167

168 of 406

Existence of inverses

  •  

168

169 of 406

Existence of inverses

  •  

169

170 of 406

Existence of inverses

  •  

170

171 of 406

Existence of inverses

  •  

171

172 of 406

Existence of inverses

  •  

172

173 of 406

Existence of inverses

  •  

173

174 of 406

Existence of inverses

  •  

174

175 of 406

Linear transformation

  •  

175

176 of 406

Linear transformation

  •  

176

177 of 406

Linear transformation

  •  

177

178 of 406

Linear transformation

  •  

178

179 of 406

Linear transformation

  •  

179

180 of 406

Linear transformation

  •  

180

181 of 406

Linear transformation

  •  

181

182 of 406

Linear transformation

  •  

182

183 of 406

Linear transformation

  •  

183

184 of 406

Linear transformation

  •  

184

185 of 406

Linear transformation

  •  

185

186 of 406

Linear transformation

  •  

186

187 of 406

Linear transformation

  •  

187

188 of 406

Linear transformation

  •  

188

189 of 406

Linear transformation

  •  

189

190 of 406

Linear transformation

  •  

190

191 of 406

Linear transformation

  •  

191

192 of 406

Linear transformation

  •  

192

193 of 406

Linear transformation

  •  

193

194 of 406

Linear transformation

  •  

194

195 of 406

Linear transformation

  •  

195

196 of 406

Linear transformation

  •  

196

197 of 406

Rotations, Projections & Reflections: Revisited

  • Rotation by θ:
    • We want to rotate the unit vectors (basis vectors) by an angle θ

197

198 of 406

Rotations, Projections & Reflections: Revisited

  •  

198

199 of 406

Rotations, Projections & Reflections: Revisited

  • Rotation by θ:

​

199

200 of 406

Rotations, Projections & Reflections: Revisited

  •  

200

201 of 406

Rotations, Projections & Reflections: Revisited

  •  

201

202 of 406

Rotations, Projections & Reflections: Revisited

    • Therefore, the projection matrix is given by

​

​

    • Projecting twice is same as projecting once!

202

203 of 406

Rotations, Projections & Reflections: Revisited

  •  

203

204 of 406

Isomorphism

  •  

204

205 of 406

Numerical problems

  •  

205

206 of 406

Numerical problems

  •  

206

207 of 406

Numerical problems

  •  

207

208 of 406

Numerical problems

  •  

208

209 of 406

Numerical problems

  •  

209

210 of 406

Numerical problems

  •  

210

211 of 406

Numerical problems

  •  

211

212 of 406

Numerical problems

  •  

212

213 of 406

Numerical problems

  •  

213

214 of 406

Numerical problems

  •  

214

215 of 406

Numerical problems

  •  

215

216 of 406

Numerical problems

  •  

216

217 of 406

Numerical problems

  •  

217

218 of 406

Orthogonalization, Eigenvalues and Eigen vectors

Unit 3

218

219 of 406

Orthogonal vectors and subspaces

  •  

219

220 of 406

Orthogonal vectors and subspaces

  • The concept of orthogonal basis is key to converting geometric constructs into algebraic calculations.
    • Examples:
    • Expressing the length of the vectors
    • Expressing distance between vectors or vector spaces
    • Expressing the projection of vectors
    • Expressing the angle between the vectors

220

221 of 406

Orthogonal vectors and subspaces

  •  

221

222 of 406

Orthogonal vectors and subspaces

  •  

222

 

223 of 406

Orthogonal vectors and subspaces

  •  

223

224 of 406

Orthogonal vectors and subspaces

  •  

224

225 of 406

Orthogonal vectors and subspaces

  •  

225

226 of 406

Orthogonal vectors and subspaces

  •  

226

227 of 406

Orthogonal vectors and subspaces

  •  

227

228 of 406

Orthogonal vectors and subspaces

  •  

228

229 of 406

Orthogonal vectors and subspaces

  •  

229

230 of 406

Orthogonal vectors and subspaces

  •  

230

231 of 406

Orthogonal vectors and subspaces

  •  

231

232 of 406

Orthogonal vectors and subspaces

  •  

232

233 of 406

Orthogonal vectors and subspaces

  •  

233

234 of 406

Orthogonal vectors and subspaces

  •  

234

235 of 406

Orthogonal vectors and subspaces

  •  

235

236 of 406

Orthogonal vectors and subspaces

236

237 of 406

Cosines and projections onto lines

  •  

237

238 of 406

Cosines and projections onto lines

  •  

238

239 of 406

Cosines and projections onto lines

  •  

239

 

 

240 of 406

Cosines and projections onto lines

  •  

240

241 of 406

Cosines and projections onto lines

  •  

241

242 of 406

Cosines and projections onto lines

  •  

242

243 of 406

Cosines and projections onto lines

  •  

243

244 of 406

Cosines and projections onto lines

  •  

244

245 of 406

Cosines and projections onto lines

  •  

245

246 of 406

Projection and least squares

  •  

246

247 of 406

Projection and least squares

  •  

247

248 of 406

Projection and least squares

  •  

248

249 of 406

Projection and least squares

  •  

249

250 of 406

Projection and least squares

  •  

250

251 of 406

Projection and least squares

  • Example (contd.):

251

252 of 406

Numerical problems

  •  

252

253 of 406

Numerical problems

  •  

253

254 of 406

Numerical problems

  •  

254

255 of 406

Numerical problems

  •  

255

256 of 406

Numerical problems

  •  

256

257 of 406

Numerical problems

  •  

257

258 of 406

Numerical problems

  •  

258

259 of 406

Numerical problems

  •  

259

260 of 406

Numerical problems

  •  

260

261 of 406

Numerical problems

  •  

261

262 of 406

Numerical problems

  •  

262

263 of 406

Numerical problems

  •  

263

264 of 406

Numerical problems

  •  

264

265 of 406

Numerical problems

  •  

265

266 of 406

Numerical problems

  •  

266

267 of 406

Numerical problems

  •  

267

268 of 406

Numerical problems

  •  

268

269 of 406

Numerical problems

  •  

269

270 of 406

Numerical problems

  •  

270

271 of 406

Numerical problems

  •  

271

272 of 406

Orthonormal bases

  •  

272

273 of 406

Orthonormal bases

  •  

273

274 of 406

Orthonormal bases

  •  

274

275 of 406

Orthonormal bases

  •  

275

276 of 406

Orthonormal bases

  •  

276

277 of 406

Orthonormal bases

  •  

277

278 of 406

Orthonormal bases

  •  

278

279 of 406

Orthonormal bases

  •  

279

280 of 406

Orthonormal bases

  •  

280

281 of 406

Gram Schmidt process

  •  

281

282 of 406

Gram Schmidt process

  •  

282

283 of 406

Gram Schmidt process

  •  

283

284 of 406

Gram Schmidt process

    • Step 3: Normalize B to get

​

​

​

    • Step 4: Find C

​

284

285 of 406

Gram Schmidt process

    • Step 5: No need to normalize C as it is already a unit vector
    • The Q matrix is now given by

285

286 of 406

QR factorization

  •  

286

287 of 406

QR factorization

  •  

287

288 of 406

Numerical problems

  •  

288

289 of 406

Numerical problems

  •  

289

290 of 406

Numerical problems

  •  

290

291 of 406

Numerical problems

  •  

291

292 of 406

Numerical problems

  •  

292

293 of 406

Numerical problems

  •  

293

294 of 406

Numerical problems

  •  

294

295 of 406

Numerical problems

  •  

295

296 of 406

Eigenvalues and Eigenvectors

  •  

296

297 of 406

Eigenvalues and Eigenvectors

  •  

297

298 of 406

Eigenvalues and Eigenvectors

  •  

298

299 of 406

Eigenvalues and Eigenvectors

  •  

299

300 of 406

Eigenvalues and Eigenvectors

  •  

300

301 of 406

Eigenvalues and Eigenvectors

  •  

301

302 of 406

Eigenvalues and Eigenvectors

  •  

302

303 of 406

Eigenvalues and Eigenvectors

  •  

303

304 of 406

Eigenvalues and Eigenvectors

  •  

304

305 of 406

Eigenvalues and Eigenvectors

  •  

305

306 of 406

Eigenvalues and Eigenvectors

  •  

306

307 of 406

Eigenvalues and Eigenvectors

  •  

307

308 of 406

Eigenvalues and Eigenvectors

  •  

308

309 of 406

Eigenvalues and Eigenvectors

  •  

309

310 of 406

Eigenvalues and Eigenvectors

  •  

310

311 of 406

Eigenvalues and Eigenvectors

  •  

311

312 of 406

Eigenvalues and Eigenvectors

  •  

312

313 of 406

Eigenvalues and Eigenvectors

  •  

313

314 of 406

Eigenvalues and Eigenvectors

  •  

314

315 of 406

Eigenvalues and Eigenvectors

  •  

315

316 of 406

Diagonalization of a matrix

  •  

316

317 of 406

Diagonalization of a matrix

​

​

​

​

​

  • Only matrices with unique eigenvalues are diagonalizable.

317

318 of 406

Diagonalization of a matrix

  •  

318

319 of 406

Diagonalization of a matrix

  •  

319

320 of 406

Diagonalization of a matrix

  •  

320

321 of 406

Diagonalization of a matrix

  •  

321

322 of 406

Diagonalization of a matrix

  •  

322

323 of 406

Diagonalization of a matrix

  •  

323

324 of 406

Diagonalization of a matrix

  •  

324

325 of 406

Diagonalization of a matrix

  •  

325

326 of 406

Diagonalization of a matrix

  •  

326

327 of 406

Diagonalization of a matrix

  •  

327

328 of 406

Diagonalization of a matrix

  •  

328

329 of 406

Diagonalization of a matrix

  •  

329

330 of 406

Singular Value Decomposition

Unit 4

Based on Chapter 7 in Linear Algebra and Its Applications by David C Lay

330

331 of 406

Symmetric matrices

  •  

331

332 of 406

Symmetric matrices

  •  

332

333 of 406

Symmetric matrices

  •  

333

334 of 406

Symmetric matrices

  •  

334

335 of 406

Symmetric matrices

  •  

335

336 of 406

Symmetric matrices

  •  

336

337 of 406

Symmetric matrices

  •  

337

338 of 406

Symmetric matrices

  •  

338

339 of 406

Spectral theorem

  •  

339

340 of 406

Spectral decomposition

  •  

340

341 of 406

Spectral decomposition

  •  

341

342 of 406

Quadratic forms

  •  

342

343 of 406

Quadratic forms

  •  

343

344 of 406

Quadratic forms

  •  

344

345 of 406

Quadratic forms

  •  

345

346 of 406

Quadratic forms

  •  

346

347 of 406

Quadratic forms

  •  

347

348 of 406

Quadratic forms

  •  

348

349 of 406

Quadratic forms

  •  

349

350 of 406

Quadratic forms

  •  

350

351 of 406

Quadratic forms

  • Geometric view of principal axes
    • If A is a diagonal matrix, the graph is in the standard position.

351

352 of 406

Quadratic forms

  • Geometric view of principal axes
    • If A is not a diagonal matrix, the graph is rotated from the standard position.

352

353 of 406

Quadratic forms

  •  

353

354 of 406

Quadratic forms

  •  

354

355 of 406

Quadratic forms

  •  

355

356 of 406

Quadratic forms

  • Classifying quadratic forms

​

356

357 of 406

Quadratic forms

  • Classifying quadratic forms

​

357

358 of 406

Positive definite matrices

  • Properties of positive definiteness

​

358

359 of 406

Positive definite matrices

  • Properties of positive definiteness

​

359

360 of 406

Positive definite matrices

360

361 of 406

Positive definite matrices

361

362 of 406

Quadratic forms

  •  

362

363 of 406

Quadratic forms

  •  

363

364 of 406

Practice problems

  •  

364

365 of 406

Practice problems

  •  

365

366 of 406

Practice problems

  •  

366

367 of 406

Practice problems

  •  

367

368 of 406

Singular value decomposition

  •  

368

369 of 406

Singular value decomposition

  •  

369

370 of 406

Singular value decomposition

  •  

370

371 of 406

Singular value decomposition

  •  

371

372 of 406

Singular value decomposition

  •  

372

373 of 406

Singular value decomposition

  •  

373

374 of 406

Singular value decomposition

  •  

374

375 of 406

Singular value decomposition

  •  

375

376 of 406

Singular value decomposition

  •  

376

377 of 406

Singular value decomposition

  •  

377

378 of 406

Singular value decomposition

​

​

​

​

    • The SVD of A is given by

378

379 of 406

Singular value decomposition

  •  

379

380 of 406

Singular value decomposition

  •  

380

381 of 406

Singular value decomposition

  •  

381

382 of 406

Singular value decomposition

  •  

382

383 of 406

Singular value decomposition

  •  

383

384 of 406

Principal component analysis

  •  

384

385 of 406

Principal component analysis

  •  

385

386 of 406

Principal component analysis

  • Example:
    • Three measurements are made on each of four individuals in a random sample from a population. The observation vectors are

​

386

387 of 406

Principal component analysis

  • Example (contd.):

​

387

388 of 406

Principal component analysis

    • The sample covariance matrix is given by

388

389 of 406

Principal component analysis

  •  

389

390 of 406

Principal component analysis

  •  

390

391 of 406

Principal component analysis

  •  

391

392 of 406

Principal component analysis

  •  

392

393 of 406

Principal component analysis

  •  

393

394 of 406

Principal component analysis

  •  

394

395 of 406

Principal component analysis

  •  

395

396 of 406

Principal component analysis

  •  

396

397 of 406

Practice problems

  • Example:
    • The following table lists the weights and heights of five boys:

​

​

​

    • Find the covariance matrix for the data.
    • Make a principal component analysis of the data to find a single size index that explains most of the variation in the data

397

398 of 406

Practice problems

  •  

398

399 of 406

Practice problems

  •  

399

400 of 406

Practice problems

  •  

400

401 of 406

Practice problems

    • See the graph below. It confirms our dimensionality reduction

401

402 of 406

Practice problems

  •  

402

403 of 406

Practice problems

  •  

403

404 of 406

Practice problems

  •  

404

405 of 406

Practice problems

    • A Landsat image with three spectral components was made of Homestead Air Force Base in Florida (after the base was hit by Hurricane Andrew in 1992). The covariance matrix of the data is shown below. Find the first principal component of the data, and compute the percentage of the total variance that is contained in this component.

405

406 of 406

Practice problems

  •  

406