Neural Networks and Fuzzy Systems

Pattern Recognition

Rizoan Toufiq

Assistant Professor

Department of Computer Science & Engineering

Rajshahi University of Engineering & Technology

Linear Classifier

• Can solve linearly separable problem.
• What happens with non-linear separable problem?

​

• How can we implement such a decision boundary in the case of real pattern data?

​

• How is the position of the separating boundary chosen?

Linear Classifier

• How can we implement such a decision boundary in the case of real pattern data?

​

• How is the position of the separating boundary chosen?

Linear Classifier

• The decision boundary defines a discriminating function f(X) of the form:

Linear Classifier

• The problem lies in actually finding a suitable weight vector that will give these results for all inputs from class A and class B

Linear Classifier

• The discriminant function using matrix algebra

Linear Classifier

• There are two parameters that control the position of the decision boundary in the pattern space-these are
• the slope of the line and
• the y-axis intercept (standard geometry of a straight line).

​

• The slope of the line in the function is actually determined by the value of the weight vector.

​

• We can see this if we consider the crossover point, or boundary condition, when the output of the classifier is zero i.e. f(X) = 0.
• We have:

Linear Classifier

• Comparing this to the equation of a straight line (y = mx + c), we can see that
• the slope of the line is controlled by the ratio of the weight values W₁ and W₂ and
• the intercept is controlled by the bias value, θ.

Linear Classifier

• What we have not shown yet is the critical part-namely finding the weight vector.
• This, unfortunately, is a not a trivial problem!

​

• It is most usually found by iterative trial and error methods that modify the weight values according to some error function.
• The error function typically compares the output of the classifier with a desired response and gives an indication of the difference between the two.

Linear Classifier

• Linear classifiers can also be used to separate more than two classes, by arranging many decision boundaries and performing several tests to satisfy the conditions for each class.

​

• As an example, in a four-class problem (A, B, C, D),the decision boundaries can be selected to test between A or BCD, if the result is not A then test for B or CD, if not B then test for C or D.

Linear Classifier

• for difficult class boundary conditions the decision surface can be split up in a piecewise fashion

Linear Classifier

• In non-linearly separable problems it is also possible to introduce the required non-linearity into the decision surface by applying a non-linear transformation to the data before it is passed to the classifier stage.

​

• This technique is described as a Ф machine and such preprocessing of pattern data before passing it to a pattern classifier is common practice.

​

• A transform is found that will map the patterns into a new coding that is capable of being classified using a linear classifier.
• The major drawback of this approach is that it can be slow

Statistical Techniques

• Discuss the concept of Bayesian classification

​

• forcing us to think harder about the statistical nature of the data that we are dealing with in pattern recognition problems.

​

• Addressing our problem statistically we can gain a very useful insight into the nature of the pattern data that we are dealing with-as well as perhaps a more intuitive feel for what makes pattern recognition problems often so difficult to solve.

Statistical Techniques

• Let G_i, I = 1, 2, …, n → list of possible classes or groups then we can define the probability of a pattern belonging to a class as P(G_i) where 0 ≤ P(G_i) ≤ 1.

​

• Conditional Probability: the probability of event X given the occurrence of even Y → P(X|Y)
• For pattern recognition, features of a pattern X = (x₁, x₂, …, x_n), then the likelihood, or probability, of pattern X belonging to a class G_i is P(G_i|X)
• Bayes’s rule assign X to a class on the following basis:
• Decide x belong to class I for

P(G_i|X) > P(G_j|X), for i = 1, 2, …, n i ≠ j

• Assign to the class that has the highest conditional probability of the vector X belonging to it.

Statistical Techniques

• We know the pattern must belong to one of n classes,
• What is the probability of obtaining that pattern vector in each of the possible class, P(X|G_i)?
• We do not know the absolute value of this probability.
• We can approximate it by using a model probability distribution
• Relation between P(G_i|X) and P(X|G_i), Bayes’s law:

​

​

​

• P(G_i) → find without too much difficulty
• P(X|G_i) → Normal distribution

Statistical Techniques

• This classifier will be linear
• by making the assumption that both the class distributions have equal covariance matrices.

Summary

Rizoan Toufiq, Assistant Professor, Dept. of CSE, RUET

17

• Linear classification occurs when classes can be separated by a single linear decision boundary.
• Classes that cannot be separated this way are termed non-linearly separable.

Read Task

Rizoan Toufiq, Assistant Professor, Dept. of CSE, RUET

18

B1: Neural Computing - An Introduction - R Beale and T Jackson, Publisher: Adam Hilger, 1990 IOP Publishing Ltd.

Chapter 2: Pattern Recognition

Query???

Rizoan Toufiq, Assistant Professor, Dept. of CSE, RUET

19