Lecture 1: �Linear Algebra

André E. Lazzaretti

UTFPR/CPGEI

Introduction

• Linear algebra is the study of vectors and certain rules to manipulate vectors.
• Types of vectors:
• Geometric vectors;
• Polynomials;
• Audio signals;
• Elements of

​

Scalars, Vectors, Matrices, and Tensors

• Scalars: A scalar is just a single number, in contrast to most of the other objects studied in linear algebra, which are usually arrays of multiple numbers.

​

​

​

• Vectors: A vector is an array of numbers. The numbers are arranged in order. We can identify each individual number by its index in that ordering.

Scalars, Vectors, Matrices, and Tensors

• Matrices: A matrix is a 2-D array of numbers, so each element is identified by two indices instead of just one.

​

​

​

​

​

​

​

​

​

​

​

• Tensors: In some cases we will need an array with more than two axes.

Multiplying Vectors and Matrices

• Matrix product:

​

​

• Element-wise product:

​

​

​

Inner Products

• Inner products <.,.> allow for the introduction of intuitive geometrical concepts, such as the length of a vector and the angle or distance between two vectors.
• A major purpose of inner products is to determine whether vectors are orthogonal to each other.
• Dot Product:

Properties

• Distributive:

​

• Associative:

​

• “Commutative”:

Norms

• Inner products and norms are closely related in the sense that any inner product induces a norm:

​

​

Norms

• L^p norm:

​

​

• A norm is any function f that satisfies:

Norms

• The L² norm, with p = 2, is known as the Euclidean norm. It is also common to measure the size of a vector: x^Tx.

​

​

​

​

• The L1 norm is commonly used in machine learning when the difference between zero and nonzero elements is very important.

Norms

• Max norm:

​

​

• Frobenius norm:

​

​

​

​

• Dot product in terms of norms:

Example

Example

Distances