Unsupervised Learning
& Reinforcement Learning
CSIR Modelling and Digital Science
Nyalleng Moorosi
Borrowed from: Dr. Vukosi Marivate - Senior Data Scientist
Machine Learning
Labels? What Labels?
2
Unsupervised Learning
“the automatic discovery of regularities in data through the use of computer algorithms and with the use of these regularities to take actions such as classifying the data into different categories” [Bishop]
3
Supervised -> Unsupervised
We have input data x, we don't have have labels, what to do?
4
Why?
5
Motivations, Real World
Customer segmentation for better donation solicitation.
6
Motivations, Real World
2 weeks of Twitter data Data collected from Gauteng, Word2Vec + tSNE
7
Motivations, Real World
Closer look at some of the Word2Vec Groupings. How can we find these clusters?
8
When?
Wikipedia
9
k-means Clustering
Given
We would like to partition data into k sets
Goal: Minimise Within-Cluster sum of squares
10
k-means Algorithm
Assign
Update
11
K-Means Algorithm
Wikipedia
12
K-Means Discussion
Pros:
To note
Uses:
Wikipedia
13
K-Means Choosing k
Within Cluster Sum of Squares
Silhouette Coefficient
14
K-means demo
http://bit.ly/phonepricedata
15
Expectation Maximisation (Gaussian Mixture Models)
Given
We would like to partition data into k sets
But, allow mixed memberships to each set/cluster, further each set will be made up of a multivariate normal distribution.
16
Expectation Maximisation
(Gaussian Mixture Models)
17
Expectation Maximisation
(Gaussian Mixture Models)
18
Expectation Maximisation (Gaussian Mixture Models)
EM Algorithm
E-Step:
M-Step:
19
Expectation Maximisation (Gaussian Mixture Models)
E-Step:
M-Step:
Source: Wikipedia
20
Expectation Maximisation (Gaussian Mixture Models)
Results
Wikipedia
21
Expectation Maximisation (Gaussian Mixture Models): Discussion
Pros:
To note:
Uses:
sklearn
22
k-means soft clustering
Minimise
Where
Source: Wikipedia
23
More Clustering
Can we just use the similarity of points to each other to cluster?
Answer: YES
24
Hierarchal Clustering
Bottom Up Clustering
Need
25
Hierarchal Clustering: �Similarity Measures
26
Hierarchal Clustering: �Linkage Criteria
How to merge?
Single Linkage
Complete Linkage
27
Hierarchal Clustering: �Visualisation
Dendogram
Source: Vukosi Marivate and Nyalleng Moorosi SA job market analysis
28
Hierarchal Clustering: �Where to cut?
29
Hierarchal Clustering: �Discussion
Pros:
To note
Uses:
30
Practical: MNIST Digit Recognition
Hand-written digits, ignore labels.
31
Practical: The news problem
You are given a large number of documents, you would like to characterise what these documents are talking about, but labelling is expensive. Can we use unsupervised learning to find out the latent structure of these documents?
32
Advanced Topics
General
Text
Graphs
33
Advanced Topics
34
Advanced Topics
35
Reinforcement Learning
Striving to solve the AI problem, one reward at a time.
36
The Apartment Domain
Source: V Marivate: IMPROVED EMPIRICAL METHODS IN REINFORCEMENT-LEARNING EVALUATION
37
Reinforcement Learing
38
Reinforcement Learning
39
Reinforcement Learning
40
Value Iteration
Initialise all Values (V’s) to zero. Then iteratively do
Dynamic Programming
41
Online Paradigm
Q Learning
e-Greedy
42
Online Paradigm
Learn while in the world:
Q Learning Algorithm:
Action choices:
43
Offline Learning
Learn from collected data:
LSTDQ Learning Algorithm
44
Offline Learning
To Note:
45
RL + Unsupervised Learning???
Source: V Marivate: IMPROVED EMPIRICAL METHODS IN REINFORCEMENT-LEARNING EVALUATION
46
RL + Unsupervised Learning???
47
RL + Unsupervised Learning???
48
RL + Unsupervised Learning???
49
RL + Unsupervised Learning???
Source: V Marivate: IMPROVED EMPIRICAL METHODS IN REINFORCEMENT-LEARNING EVALUATION
50
Bibliography
51
Always looking for:
Students, Collaborations, Shared Passions
so get in touch
@vukosi
vmarivate@csir.co.za
52