Machine Learning - �Basic Principles & Practice�5. From Lazy Learning to �Eager Learning

Cong Li 李聪

机器学习 - 基础原理与实践

5. 从临阵磨枪到积极学习

Lazy Learning 临阵磨枪

• Rote Learning & Nearest Neighbor死记硬背和最近邻
• Memorize all the training data in learning 在学习时记住所有的训练数据
• Nothing else 没有其他的了
• It’s Lazy 这很懒惰
• Meaningful work done in classification 有意义的事情都是分类时做的

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Lazy Learning Characteristics�临阵磨枪的特点

• No Explicit Knowledge Learned 没有学习到显式的知识
• Classification complexity depends on data 分类的复杂程度由数据决定
• Can either be good or bad �这有时是好事，有时是坏事
• Slow in Classification 分类很慢
• Have to go through all the data 必须遍历所有的数据

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Eager Learning 积极学习

• Learn beyond Memorizing �超越记忆的学习
• Do more in training 训练时做得更多
• Learn a certain explicit knowledge representation 学习到了特定的显式知识表示
• Let’s Start from Linear Classifiers�让我们从线性分类器开始

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Explicit Knowledge 显式的知识

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

2D Example 二维的示例

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Attribute 1

​

属性1

Attribute 2 属性2

2D Example 二维的示例

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Attribute 1

​

属性1

Attribute 2 属性2

Linear Classifier 线性分类器

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Biological View 生物学的视角

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Neuron 神经细胞

Linear Classifier as a Neuron�作为神经细胞的线性分类器

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

?

How to Learn? 怎么学？

• Paradigm: Online Learning �模式：在线学习
• Each time, classify a certain data item w/ the current classifier 每一时刻，用当前分类器对一个数据进行分类
• Compare the classification result w/ the ground truth 比较分类结果和真实类别
• Adjust the classifier 调整分类器
• Perceptron Algorithm 感知器算法

​

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Perceptron Algorithm�感知器算法

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Batch Learning 批量学习

• Typical Case: Not the Online One�典型的情况：并非在线学习
• Repeatedly loop over the dataset�反复对数据集进行循环
• Stop Conditions 结束条件
• Stop learning if no errors in a batch�如果一轮中没有出错，就可以结束学习
• Otherwise? 否则呢？
• Make it simple, stop after a maximum number of rounds 简单点：到达最大轮数终止

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Practice 5.1 实践5.1

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Practice time: use perceptron on the 1-attribute & 2-attribute synthetic problems

实践时刻：将感知器用于一个属性和两个属性的人造问题

Compare the results w/ those from nearest neighbor

和最近邻的结果进行比较

Result 结果

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Perceptron performs stably, but why?

感知器表现得很稳定，这是为什么？

Do You Still Remember?�你还记得吗？

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

No assumption, no learning!

无假设，不学习！

Now where is the assumption?

那么假设在哪里？

Assumption 假设

• Assumption 假设
• Classification can be done w/ a hyperplane 能用一个超平面完成分类
• Perceptron Algorithm 感知器算法
• If there exists a hyperplane separating the 2 classes, perceptron guarantees to find such a hyperplane�如果存在一个划分两类的超平面，那么感知器也能找到一个这样的超平面

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Synthetic Problem 人造问题

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Assumption Holds 假设成立

• This Synthetic Problem 该人造问题
• Assumption holds 假设成立
• Hyperplane separates the data�超平面分割了数据
• How about Real World Problem?�真实的问题如何？
• Like word sense disambiguation for ‘interest’ 例如“interest”的词义消歧
• Wait, this is a multi-class problem�等一下，这个是多类问题

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Multi-class Linear Classifier �多类线性分类器

• Linear Classifier 线性分类器
• Hyperplane splits data into 2 parts�超平面将数据一分为二
• Given More than 2 Classes �当多于两个类别时
• Construct multiple linear classifiers �创建多个线性分类器
• Each classifier: one vs. rest�每个分类器：一对其余

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

One vs. Rest 一对其余

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Classifier

分类器

One vs. Rest 一对其余

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Classifier

分类器

One vs. Rest 一对其余

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Classifier

分类器

Compete w/ the Score 比较得分

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Classifier

分类器

Classifier

分类器

Classifier

分类器

Data 数据

Compete 比较

Practice 5.2 实践5.2

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Practice time: use perceptron on word sense disambiguation

实践时刻：将感知器用于词义消歧

Compare the results w/ those from nearest neighbor

和最近邻的结果进行比较

Result 结果

Machine Learning – Basic Principles & Practice: 5. From Lazy Learning to Eager Learning

机器学习 – 基础原理与实践：5. 从临阵磨枪到积极学习

Perceptron improves the accuracy a lot

感知器大幅提升了准确率

The End