�Review of the key concepts of session no. 19�

1

• Cross-Validation is a model evaluation strategy, not a performance measure.
• It answers the question: “How well will my model perform on unseen data?”
• By splitting data into multiple train–test sets, it reduces the risk of overfitting (where the model only memorizes training data).
• The most widely used version is k-Fold Cross-Validation, with Stratified k-Fold being especially important for classification problems with imbalanced datasets.
• Classification Accuracy is a performance metric, not an evaluation method.
• It answers the question: “How often does my model predict correctly?”
• It is easy to interpret and compare across models.

​

Decision Trees

2

• A decision tree is a hierarchical, tree-like structure used in decision-making and data analysis.
• A Decision Tree is a type of supervised machine learning algorithm used for both classification and regression tasks.
• It works by splitting data into subsets based on the value of input features, building a tree-like structure where each internal node represents a decision (or test), each branch represents the outcome of that decision, and each leaf node represents the final output or prediction.

​

Decision Trees

3

Use of Decision Tree in Machine Learning

• Decision Trees are widely used in supervised learning for both:
• Classification – Predicting a categorical output (e.g., spam or not spam).
• Regression – Predicting a continuous output (e.g., price of a house).
• Gini Impurity (Classification)
• Entropy/Information Gain (Classification)
• Mean Squared Error (Regression)

​

Decision Trees

4

Key Components of a Decision Tree:

• Root Node: The topmost node, representing the entire dataset. It gets split into two or more homogeneous sets.
• Decision Nodes: Internal nodes that split the data further based on a condition.
• Leaf Nodes (Terminal Nodes): Final nodes that provide the prediction or outcome.
• Branches: Represent the outcome of a decision or test.

​

Decision Trees

5

Decision Trees

Galgotias University

6

Learning Activity 1�Pen and Paper�

7

Sample Cricket Dataset for Decision Tree

Learning Activity 1�Pen and Paper�

8

Learning Activity 1�Pen and Paper�

9

Learning Activity 1�Pen and Paper�

10

Learning Activity �Pen and Paper�

11

Gini Impurity

Learning Activity �Pen and Paper�

12

Gini Impurity

Answer:

Learning Activity �Pen and Paper�

13

Regression Task

Learning Activity �Pen and Paper�

14

Regression Task

Applications of Decision Trees��

15

• Loan Approval in Banking: A bank needs to decide whether to approve a loan application based on customer profiles.
• Input features include income, credit score, employment status, and loan history.
• The decision tree predicts loan approval or rejection, helping the bank make quick and reliable decisions.
• Medical Diagnosis: A healthcare provider wants to predict whether a patient has diabetes based on clinical test results.
• Features like glucose levels, BMI, and blood pressure are used to make a decision tree.
• Tree classifies patients into diabetic or non-diabetic, assisting doctors in diagnosis.

Learning Activity �Pen and Paper�

16

Sample Student Dataset for Decision Tree

Learning Activity �Pen and Paper�

17

Sample Student Dataset for Decision Tree

Calculate the following metrics for your dataset:

• Classification:
• Gini Impurity
• Information Gain (based on Gini or Entropy)
• Regression:
• MSE (Mean Squared Error)
• R² Score (Coefficient of Determination)

Summary

Galgotias University

18

• A Decision Tree is a supervised learning algorithm used for classification and regression tasks.
• It splits the data into subsets based on feature values, forming a tree-like structure.
• For classification, it uses criteria like Information Gain or Gini Impurity to decide splits.
• For regression, it uses metrics like Mean Squared Error (MSE) to minimize prediction error.
• Decision Trees are easy to interpret and handle both numerical and categorical data.

​

19

Ensure attainment of LOs in alignment to the learning activities: outcomes (

Outcome 2:

Interpret real-world data using descriptive statistics.

.

Outcome 1:

Students will be able to describe what a Decision Tree is and how it is used to make decisions in classification problems.

​

Outcome 2:

Students will be able to solve simple classification problems by building and using a Decision Tree.

​

��Discussion on the post session activities��

Galgotias University

20

Activity: create a decision tree for given dataset

• .

​

21

Information to next topic of the course

���

• Inductive Bias and Inference
• Introduction
• Inductive Bias
• Inference in Learning
• Role of Inductive Bias in Inference
• Case Examples

Review and Reflection from students

22