Reconhecimento de Padrões (RPD-0041)�

Prof. André E. Lazzaretti

lazzaretti@utfpr.edu.br

https://sites.google.com/site/andrelazzaretti/graduate-courses/reconhecimento-de-padr%C3%B5es-cpgei/cronograma-2026

Lecture 3b: A Tour of Machine Learning Classifiers Using scikit-learn - Support Vector Machine

General Idea

​

• Select the model “type”

​

• Select a loss function

​

• Select as optimization method

​

• Find the solution

Support Vector Machine

• Which of the two linear classifiers below would you choose?

Intuitively, the one that provides the largest separation margin!

Maximum Margin Intuition

SVM optimization

The left side of the preceding equation can then be interpreted as the distance between the positive and negative hyperplane, which is the so-called margin that we want to maximize. Now, the objective function of the SVM becomes the maximization of this margin by:�

Support Vector Machine

Support Vector Machine

λ_i non-zero →

support vectors

Support Vector Machine

• Non-linearly separable case:

Three cases:

• Patterns outside the margin, correctly classified;

​

• Patterns inside the margin, correctly classified:

​

• Patterns inside the margin, incorrectly classified:

​

Support Vector Machine

• The optimization problem could be written with the objective of maximizing the margin, penalizing patterns inside the margin and incorrectly classified patterns:

https://math.stackexchange.com/questions/1021586/about-the-slack-variable-for-hinge-loss-svm

Dealing with a nonlinearly separable case�using slack variables

• Soft-margin classification:�

Support Vector Machine

Influence of parameter C:

What does this mean?

Relationship with regularization!

SVM using scikit-learn

Codes