Active Learning

Segmentation

• Segment images into regions with different semantic categories.
• These semantic regions label and predict objects at the pixel level

​

​

2

Image from http://d2l.ai/

Microstructure Segmentation

• Structure-property linkages

​

​

3

Juwon Na, Se-Jong Kim, Seong-Hoon Kang, Heekyu Kim and Seungchul Lee*, 2022, "A Unified Microstructure Segmentation Approach via Human-In-The-Loop Machine Learning," Acta Materialia

Given input

Fully supervised

Weakly-supervised

Microstructure Segmentation

• Structure-property linkages

​

​

4

Juwon Na, Se-Jong Kim, Seong-Hoon Kang, Heekyu Kim and Seungchul Lee*, 2022, "A Unified Microstructure Segmentation Approach via Human-In-The-Loop Machine Learning," Acta Materialia

Given input

Fully supervised

Weakly-supervised

Weakly Supervised Learning (WSL): Phase Segmentation

5

Segmentation result

Scribble annotation

Juwon Na, Se-Jong Kim, Seong-Hoon Kang, Heekyu Kim and Seungchul Lee*, 2022, "A Unified Microstructure Segmentation Approach via Human-In-The-Loop Machine Learning," Acta Materialia

Weakly Supervised Learning (WSL): Phase Segmentation

6

Segmentation result

Scribble annotation

Juwon Na, Se-Jong Kim, Seong-Hoon Kang, Heekyu Kim and Seungchul Lee*, 2022, "A Unified Microstructure Segmentation Approach via Human-In-The-Loop Machine Learning," Acta Materialia

WSL + Active Learning

7

Iteration 1

Iteration 2

Iteration 3

Annotated Image

Segmentation

Juwon Na, Se-Jong Kim, Seong-Hoon Kang, Heekyu Kim and Seungchul Lee*, 2022, "A Unified Microstructure Segmentation Approach via Human-In-The-Loop Machine Learning," Acta Materialia

WSL + Active Learning

8

Iteration 1

Iteration 2

Iteration 3

Annotated Image

Segmentation

Juwon Na, Se-Jong Kim, Seong-Hoon Kang, Heekyu Kim and Seungchul Lee*, 2022, "A Unified Microstructure Segmentation Approach via Human-In-The-Loop Machine Learning," Acta Materialia

WSL + Active Learning

9

Iteration 1

Iteration 2

Iteration 3

Annotated Image

Segmentation

Juwon Na, Se-Jong Kim, Seong-Hoon Kang, Heekyu Kim and Seungchul Lee*, 2022, "A Unified Microstructure Segmentation Approach via Human-In-The-Loop Machine Learning," Acta Materialia

Active Learning

10

Data set

Test set

Train set 1

Model 1

Active Learning

11

Data set

Test set

Train set 1

Model 1

Model 2

Train set 2

Model 3

Train set 3

Model n

Prediction

Train set n

Active Learning: Two Purposes

1. Active Learning to Build Good Surrogate Model (DOE)
• Sampling unlabeled data that is expected to most significantly enhance performance of AI models
• Active learning with Bayesian optimization (BO)
2. Active Learning to Find Better Solution (Optimization)
• Identify unlabeled data that is expected to have highest label values

​

​

​

12

1) Active Learning to Build Good AI Model

13

Improving Model Efficiency

​

​

14

Improving Model Efficiency

​

​

15

Improving Model Efficiency

​

​

16

Improving Model Efficiency

17

Improving Model Efficiency

​

​

18

• Looks very natural and easy
• In high dimensional space, it is hard to imagine or visualize
• Requires mathematical methods

Improving Model Efficiency

​

​

19

unlabeled

• Looks very natural and easy
• In high dimensional space, it is hard to imagine or visualize
• Requires mathematical methods

Improving Model Efficiency

​

​

20

unlabeled

• Looks very natural and easy
• In high dimensional space, it is hard to imagine or visualize
• Requires mathematical methods

Improving Model Efficiency

​

​

​

​

​

​

​

​

​

​

​

​

• Selects the most informative data points to minimize labeling efforts while maximizing learning efficiency.

​

21

unlabeled

• Looks very natural and easy
• In high dimensional space, it is hard to imagine or visualize
• Requires mathematical methods

Which Unlabeled Data Should We Sample?

22

Which Unlabeled Data Should We Sample?

• Basic idea

23

Which Unlabeled Data Should We Sample?

• Basic idea

24

Uncertainty-based Sampling

• Key idea: Sample the unlabeled data that the AI model is least certain how to label

​

• Measures to quantify uncertainty
• Least confidence
• Select the data with the lowest maximum probability
• Margin sampling
• Select the data that has the smallest difference between the first and second most probable labels

​

​

​

​

​

​

25

AI model’s prediction on two unlabeled dataset

Uncertainty-based Sampling

• Key idea: Sample the unlabeled data that the AI model is least certain how to label

​

• Measures to quantify uncertainty
• Least confidence
• Select the data with the lowest maximum probability
• Margin sampling
• Select the data that has the smallest difference between the first and second most probable labels

​

​

​

​

​

​

26

AI model’s prediction on two unlabeled dataset

Other Sampling Methods

• Providing labels for unlabeled data that the AI model is least certain is not the only way to improve AI models’ performance

27

Query-By-Committee (QBC)

Core-Set

Expected Model Change

• Involve committee of models that are trained on the same training dataset
• Instead of relying on single AI model, use multiple AI models

​

• For classification
• Select the data that committees most disagree

​

• For regression
• Select the data with maximum variance

​

Other Sampling Methods

• Providing labels for unlabeled data that the AI model is least certain is not the only way to improve AI models’ performance

28

Query-By-Committee (QBC)

Core-Set

• Involve committee of models that are trained on the same training dataset
• Instead of relying on single AI model, use multiple AI models

​

• For classification
• Select the data that committees most disagree

​

• For regression
• Select the data with maximum variance

​

Expected Model Change

Other Sampling Methods

• Providing labels for unlabeled data that the AI model is least certain is not the only way to improve AI models’ performance

29

Query-By-Committee (QBC)

Core-Set

• Select the data that would impact the most significant change to the current model if we knew its label

​

• Significant ‘change’: Estimated as the gradient of the loss with respect to a candidate example

​

Expected Model Change

Other Sampling Methods

• Providing labels for unlabeled data that the AI model is least certain is not the only way to improve AI models’ performance

30

Query-By-Committee (QBC)

Core-Set

• Find ‘Core-Set’ that can cover the entire unlabeled data and query them

​

• Core-Set: A set of data that can represent all data points within a distance of 𝛿 in the latent space

​

Expected Model Change

Active Learning

• Let’s selectively label only the data that can improve the performance of AI models
• Minimizing the number of labelling process while…
• Maximizing the performance of AI model
• Iteratively sample unlabeled data for labeling and update the training database
• As iterations are repeated, the performance of the AI model improves

​

​

​

31

Step 1

• Train AI model with training dataset

32

Step 2

• Predict labels of unlabeled data with trained AI model

33

Step 3

• Sample some of unlabeled data that would improve prediction performance of AI model
• Number of unlabeled data to be sampled is up to the user
• Various methods to measure which unlabeled data is effective in enhancing model performance exist

34

Step 4

• Perform experiments or hire experts to label the sampled unlabeled data

35

Step 5

• Add newly labeled data into the training dataset
• Repeat step 1 to 5 until AI model reach required performance

36

Let’s Talk about Ensembles

37

Ensemble with Different Models

• Ensembles: collections of predictors
• Combine predictions to improve performance

​

​

​

​

​

​

​

​

​

​

38

Ensemble with Different Training Sets

• Bagging
• parallel

​

​

39

Ensemble with Different Training Sets

• Boosting
• sequential

40

Data set

Test set

Train set 1

Model 1

Model 2

Train set 2

Model 3

Train set 3

Model n

Prediction

Train set n

Space vs. Time

41

2) Active Learning for Optimization

42

Objective

• Similar to traditional active learning in the sense that it involves sampling from unlabeled dataset, but…
• Objective: Sample unlabeled data that is expected to have highest label values
• Example) Sampling unlabeled process parameters for maximizing productivity

43

Purpose of Sampling

Which One is Better?

44

Which One is Better?

45

Which One is Better?

46

Which One is Better?

47

Active Learning for Optimization

• Two important components
• Surrogate model
• Return predicted labels and uncertainties of such predictions
• Utility function
• Use the results of the surrogate model to measure which unlabeled data is more likely to have higher label values than currently known label values

​

48

Surrogate Model

• “Surrogate” time-consuming and costly labeling processes by predicting the label values for each unlabeled data
• It returns predicted labels and uncertainties of such predictions
• Uncertainty values are necessary information for the utility function and, therefore, surrogate model is usually a probabilistic model

49

Surrogate Model

• “Surrogate” time-consuming and costly labeling processes by predicting the label values for each unlabeled data
• It returns predicted labels and uncertainties of such predictions
• Uncertainty values are necessary information for the utility function and, therefore, surrogate model is usually a probabilistic model

50

Surrogate Model

• “Surrogate” time-consuming and costly labeling processes by predicting the label values for each unlabeled data
• It returns predicted labels and uncertainties of such predictions
• Uncertainty values are necessary information for the utility function and, therefore, surrogate model is usually a probabilistic model

51

Surrogate Model

• “Surrogate” time-consuming and costly labeling processes by predicting the label values for each unlabeled data
• It returns predicted labels and uncertainties of such predictions
• Uncertainty values are necessary information for the utility function and, therefore, surrogate model is usually a probabilistic model

52

Surrogate Model

• “Surrogate” time-consuming and costly labeling processes by predicting the label values for each unlabeled data
• Widely used probabilistic models: Gaussian process regression (GPR) & Bayesian neural network (BNN)

53

Gaussian Process Regression

• GPR derives a distribution of functions that can map input to label values based on training data
• Herein, the forms of these functions are defined by a kernel function

Bayesian Neural Network

Surrogate Model

• “Surrogate” time-consuming and costly labeling processes by predicting the label values for each unlabeled data
• Widely used probabilistic models: Gaussian process regression (GPR) & Bayesian neural network (BNN)

54

Gaussian Process Regression

• BNN treat weights of neural network as distributions rather than fixed values
• This allows the network to express uncertainty in its predictions by considering a range of possible weights

Bayesian Neural Network

Step 1

• Train the surrogate model with training dataset

55

Step 2

• Predict labels of unlabeled data with trained surrogate model
• Obtain uncertainties of those predictions as well

56

Step 3

• Use utility function to sample some of unlabeled data that would possess higher label values than currently known labels
• Number of unlabeled data to be sampled is up to the user
• Various utility functions to measure which unlabeled data is more likely to have higher label values than currently known label values exist

57

Step 4

• Perform experiments or hire experts to label the sampled unlabeled data

58

Step 5

• Check if there are any newly labeled data with label values higher than the previous ones and record those values
• Add newly labeled data into the training dataset
• Repeat step 1 to 5 until the highest label value reach predefined condition

59

How Utility Functions Decide Which Unlabeled Data to Sample?

60

Utility Function: Two Important Ideas

• Exploitation: Searching the neighborhood of the point with the maximum function value among the points investigated so far in the next step
• Exploration: Searching the neighborhood of the point with the maximum standard deviation
• Strategies
• Focusing solely on exploitation can lead to neglecting data from unexplored areas
• Conversely, focusing solely on exploration can lead to neglecting data from areas that have demonstrated high label values
• Using combination of exploration and exploitation as an utility function is a effective strategy to mitigate the trade-off

​

61

Utility Function

• Utility function strategically selects unlabeled data that is more likely to have higher label values than currently known label values using exploitation, exploration, or a combination of both

62

Probability of Improvement

Expected Improvement

• Purely exploitative strategy

​

​

​

​

​

​

​

​

​

Upper Confidence Bound

Utility Function

• Utility function strategically selects unlabeled data that is more likely to have higher label values than currently known label values using exploitation, exploration, or a combination of both

63

Probability of Improvement

Expected Improvement

Upper Confidence Bound

Utility Function

• Utility function strategically selects unlabeled data that is more likely to have higher label values than currently known label values using exploitation, exploration, or a combination of both

64

Probability of Improvement

Expected Improvement

• Balancing mean prediction and uncertainty

​

​

​

​

​

• Favoring uncertainty under the assumption that higher uncertainty hides a potentially higher reward

Upper Confidence Bound

Utility Function

• Utility function strategically selects unlabeled data that is more likely to have higher label values than currently known label values using exploitation, exploration, or a combination of both

65

Probability of Improvement

Expected Improvement

Upper Confidence Bound

Utility Function

• Utility function strategically selects unlabeled data that is more likely to have higher label values than currently known label values using exploitation, exploration, or a combination of both

66

• Fuse exploration strategy into probability of improvement (PI)
• Weighting PI value by the difference between the current max value and the mean value
• Probability of obtaining data with larger label value than the existing points is important, but it is also very important how large a value is obtained

​

Probability of Improvement

Expected Improvement

Upper Confidence Bound

Exploitation

Utility Function

• Utility function strategically selects unlabeled data that is more likely to have higher label values than currently known label values using exploitation, exploration, or a combination of both

67

• Fuse exploration strategy into probability of improvement (PI)
• Weighting PI value by the difference between the current max value and the mean value
• Probability of obtaining data with larger label value than the existing points is important, but it is also very important how large a value is obtained

​

Probability of Improvement

Expected Improvement

Upper Confidence Bound

Exploration

Expected Improvement (EI)

68

Expected Improvement (EI)

69

Expected Improvement (EI)

70

Expected Improvement (EI)

71

Expected Improvement (EI)

72

Expected Improvement (EI)

73

Expected Improvement (EI)

74

Expected Improvement (EI)

75

Expected Improvement (EI)

76

Summary

• Main objectives of active learning with Bayesian optimization
• Maximize the label value by labelling previously unlabeled data
• Minimize the number of labelling process

​

• Key idea: Sample unlabeled data that is expected to most significantly enhance label values

​

• Choose right active learning framework to use

​

​

77