Gramin (ACS) Mahavidyalaya Vasantnagar , � Kotgyal � Dep. Of Mathematics � (B.Sc.TY)� Sem-5^th Paper no.–14^th � (Linear Algebra)� By- Dr.S.S.Zampalwad

(HOD)

Department of Mathematics

Gramin ACS college Vasantnagar, kotgyal

Tq: Mukhed Dist: Nanded

Inner Product Spaces

Inner product

Linear functional

Adjoint

• Assume F is a subfield of R or C.
• Let V be a v.s. over F.
• An inner product on V is a function�VxV -> F i.e., a,b in V -> (a|b) in F s.t.
• (a) (a+b|r)=(a|r)+(b|r)
• (b)( ca|r)=c(a|r)
• (c ) (b|a)=(a|b)^-
• (d) (a|a) >0 if a ≠0.
• Bilinear (nondegenerate) positive.
• A ways to measure angles and lengths.

• Examples:
• Fⁿ has a standard inner product.
• ((x₁,..,x_n)|(y₁,…,y_n)) =
• If F is a subfield of R, then = x₁y₁+…+x_ny_n.
• A,B in F^nxn.
• (A|B) = tr(AB*)=tr(B*A)
• Bilinear property: easy to see.
• tr(AB*)=

• (X|Y)=Y*Q*QX, where X,Y in F^nx1, Q nxn invertible matrix.
• Bilinearity follows easily
• (X|X)=X*Q*QX=(QX|QX)_std ≥0.
• In fact almost all inner products are of this form.
• Linear T:V->W and W has an inner product, then V has “induced” inner product.
• p_T(a|b):= (Ta|Tb).

• (a) For any basis B={a₁,…,a_n}, there is an inner-product s.t. (a_i|a_j)=δ_ij.
• Define T:V->Fⁿ s.t. a_i -> e_i.
• Then p_T(a_i|a_j)=(e_i|e_j)= δ_ij.
• (b) V={f:[0,1]->C| f is continuous }.
• (f|g)= ∫₀¹ fg^- dt for f,g in V is an inner product.
• T:V->V defined by f(t) -> tf(t) is linear.
• p_T(f,g)= ∫₀¹tftg^- dt=∫₀¹t²fg^- dt is an inner product.

• Polarization identity: Let F be an imaginary field in C.
• (a|b)=Re(a|b)+iRe(a|ib) (*):
• (a|b)=Re(a|b)+iIm(a|b).
• Use the identity Im(z)=Re(-iz) .
• Im(a|b)=Re(-i(a|b))=Re(a|ib)
• Define ||a|| := (a|a)^1/2 norm
• ||a±b||²=||a||²±2Re(a|b)+||b||^{2 (**)}.
• (a|b)=||a+b||²/4-||a-b||²/4+i||a+ib||²/4�-i||a-ib||²/4. (proof by (*) and (**).)
• (a|b)= ||a+b||²/4-||a-b||²/4 if F is a real field.

• When V is finite-dimensional, inner products can be classified.
• Given a basis B={a₁,…,a_n} and any inner product ( | ): �(a|b) = Y*GX for X=[a]_B, Y=[b]_B
• G is an nxn-matrix and G=G*, X*GX>0 for any X, X≠0.
• Proof: (->) Let G_jk=(a_k|a_j).

​

• G=G*: (a_j|a_k)=(a_k|a_j)^-. G_kj=G_jk^-.
• X*GX =(a|a) > 0 if X≠0.
• (G is invertible. GX≠0 by above for X≠0.)
• (<-) X*GY is an inner-product on F^nx1.
• (a|b) is an induced inner product by a linear transformation T sending a_i to e_i.

​

• Recall Cholesky decomposition: Hermitian positive definite matrix A = L L*. L lower triangular with real positive diagonal. (all these are useful in appl. Math.)

​

Inner product spaces

• Definition: An inner product space �(V, ( | ))
• F⊂R -> Euclidean space
• F⊂C -> Unitary space.
• Theorem 1. V, ( | ). Inner product space.
1. ||ca||=|c|||a||.
2. ||a|| > 0 for a≠0.
3. |(a|b)| ≤||a||||b|| (Cauchy-Schwarz)
4. ||a+b|| ≤||a||+||b||

• Proof (ii)

​

​

​

• Proof (iii)

• In fact many inequalities follows from Cauchy-Schwarz inequality.
• The triangle inequality also follows.
• See Example 7.
• Example 7 (d) is useful in defining Hilbert spaces. Similar inequalities are used much in analysis, PDE, and so on.
• Note Example 7, no computations are involved in proving these.

• On inner product spaces one can use the inner product to simplify many things occurring in vector spaces.
• Basis -> orthogonal basis.
• Projections -> orthogonal projections
• Complement -> orthogonal complement.
• Linear functions have adjoints
• Linear functionals become vector
• Operators -> orthogonal operators and self adjoint operators (we restrict to )

Orthogonal basis

• Definition:
• a,b in V, a⊥b if (a|b)=0.
• The zero vector is orthogonal to every vector.
• An orthogonal set S is a set s.t. all pairs of distinct vectors are orthogonal.
• An orthonormal set S is an orthogonal set of unit vectors.

• Theorem 2. An orthogonal set of nonzero-vectors is linearly independent.
• Proof: Let a₁,…,a_m be the set.
• Let 0=b=c₁a₁+…+c_ma_m.
• 0=(b,a_k)=(c₁a₁+…+c_ma_m, a_k )=c_k(a_k |a_k )
• c_k=0.
• Corollary. If b is a linear combination of orthogonal set a₁,…,a_m of nonzero vectors, then b=∑_k=1^m ((b|a_k)/||a_k||²) a_k
• Proof: See above equations for b≠0.

• Gram-Schmidt orthogonalization:
• Theorem 3. b₁,…,b_n in V independent. Then one may construct orthogonal basis a₁,…,a_n s.t. {a₁,…,a_k} is a basis for <b₁,…,b_k> for each k=1,..,n.
• Proof: a₁ := b₁. a₂=b₂-((b₂|a₁)/||a₁||²)a₁,…,
• Induction: {a₁,..,a_m} constructed and is a basis for < b₁,…,b_m>.
• Define

​

• Then

​

​

​

• Use Theorem 2 to show that the result {a₁,…,a_m+1} is independent and hence is a basis of <b₁,…,b_m+1>.

​

• See p.281, equation (8-10) for some examples.
• See examples 12 and 13.

Best approximation, Orthogonal complement, Orthogonal projections

• This is often used in applied mathematics needing approximations in many cases.
• Definition: W a subspace of V. b in W. Then the best approximation of b by a vector in W is a in W s.t. �||b-a|| ≤ ||b-c|| for all c in W.
• Existence and Uniqueness. (finite-dimensional case)

• Theorem 4: W a subspace of V. b in V.
• (i). a is a best appr to b <-> b-a ⊥ c for all c in W.
• (ii). A best appr is unique (if it exists)
• (iii). W finite dimensional.�{a₁,..,a_k} any orthonormal basis. ����is the best approx. to b by vectors in W.

• Proof: (i)
• Fact: Let c in W. b-c =(b-a)+(a-c). �||b-c||²=||b-a||²+2Re(b-a|a-c)+||a-c||²(*)
• (<-) b-a ⊥W. If c ≠a, then �||b-c||²=||b-a||+||a-c||² > ||b-a||². �Hence a is the best appr.
• (->) ||b-c||≥||b-a|| for every c in W.
• By (*) 2Re(b-a|a-c)+||a-c||² ≥0
• <-> 2Re(b-a|t)+||t||² ≥0 for every t in W.
• If a≠c, take t =

​

​

​

• This holds <-> (b-a|a-c)=0 for any c in W.
• Thus, b-a is ⊥ every vector in W.
• (ii) a,a’ best appr. to b in W.
• b-a ⊥ every v in W. b-a’ ⊥ every v in W.
• If a≠a’, then by (*)�||b-a’||²=||b-a||²+2Re(b-a|a-a’)+||a-a’||².�Hence, ||b-a’||>||b-a||.
• Conversely, ||b-a||>||b-a’||.
• This is a contradiction and a=a’.

• (iii) Take inner product of a_k with

​

​

​

• This is zero. Thus b-a ⊥ every vector in W.

Orthogonal projection

• Orthogonal complement. S a set in V.
• S^⊥ :={v in V| v⊥w for all w in S}.
• S^⊥ is a subspace. V^⊥={0}.
• If S is a subspace, then V=S⊕ S^⊥ and�(S^⊥) ^⊥=S.
• Proof: Use Gram-Schmidt orthogonalization to a basis {a₁,…,a_r,a_r+1,…,a_n} of V where {a₁,…,a_r} is a basis of V.

​

• Orthogonal projection: E_W:V->W. �a in V -> b the best approximation in W.
• By Theorem 4, this is well-defined for any subspace W.
• E_W is linear by Theorem 5.
• E_W is a projection since E_{W °}E_W(v)= E_W(v).

• Theorem 5: W subspace in V. E orthogonal projection V->W. Then E is an projection and �W^⊥=nullE and V=W⊕W^⊥.
• Proof:
• Linearity:
• a,b in V, c in F. a-Ea, b-Eb ⊥ all v in W.
• c(a-Ea)+(b-Eb)=(ca+b)-(cE(a)+E(b)) ⊥ all v in W.
• Thus by uniqueness E(ca+b)=cEa+Eb.
• null E ⊂ W^⊥ : If b is in nullE, then b=b-Eb is in W^⊥.
• W^⊥ ⊂ null E: If b is in W^⊥, then b-0 is in W^⊥ and 0 is the best appr to b by Theorem 4(i) and so Eb=0.
• Since V=ImE⊕nullE, we are done.

• Corollary: b-> b-E_wb is an orthogonal projection to W^⊥. I-E_w is an idempotent linear transformation; i.e., projection.
• Proof: b-> b-E_wb is in W^⊥ by Theorem 4 (i).
• Let c be in W^⊥. b-c=Eb+(b-Eb-c).
• Eb in W, (b-Eb-c) in W^⊥.
• ||b-c||²=||Eb||²+||b-Eb-c||²≥||b-(b-Eb)||² and�> if c ≠b-Eb.
• Thus, b-Eb is the best appr to b in W^⊥.

​

Bessel’s inequality

• {a₁,…,a_n} orthogonal set of nonzero vectors. Then

​

​

​

• = <->

​