1 of 33

Data Science and Machine Learning: Overview

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

Training Material

​

2 of 33

What is Data Science?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Data science combines math and statistics, specialized programming, advanced analytics, artificial intelligence (AI), and machine learning with specific subject matter expertise to uncover actionable insights hidden in an organization’s data. These insights can be used to guide decision making and strategic planning.

Source: IBM Data Science

3 of 33

Data Science Tools

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Data scientists rely on popular programming languages to conduct exploratory data analysis and statistical regression.
    • R
      • An open source programming language and environment for developing statistical computing and graphics.
    • Python
      • It is a dynamic and flexible programming language. The Python includes numerous libraries, such as NumPy, Pandas, Matplotlib, for analyzing data quickly.

4 of 33

What is Machine Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Machine learning, considered a subfield of artificial intelligence, is a method of data analysis that automates the construction of analytical models. It employs algorithms that learn iteratively from data, thereby enabling computers to discover insights without needing explicit programming to specify where to look.

Source: IBM Data Science

5 of 33

How do we use Machine Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Split your data into two sets: the training set and the test set.
  • 1. Train a Model:
    • Feed the training set to a learning algorithm and make a model.

Source: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow

6 of 33

How do we use Machine Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • 2. Evaluate a Model:
    • Use the test set and put your model in production.

Source: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow

7 of 33

Why use Machine Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • A ML program is much shorter, easier to maintain, and more accurate.
  • Perfect for problems that either are too complex for traditional approaches or have no known algorithm.
    • Example: Speech recognition.
  • ML can help humans learn.
    • Example: Spam filter can reveal the list of words and combinations of words that it believes are the best predictors of spam.

8 of 33

Types of Machine Learning Systems

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Whether or not they are trained with human supervision:
    • Supervised
    • Unsupervised
    • Reinforcement Learning

9 of 33

What is Supervised Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Supervised learning is a type of machine learning where the model is trained on a labeled dataset.
    • Input data (features)
  • This type of learning is often used in tasks such as regression (predicting a continuous output) and classification (predicting discrete categories).
    • Examples include predicting house prices, or classifying emails as spam or not spam.

10 of 33

Machine Learning - Supervised Learning

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Classification
    • Example: Spam filter (spam or not spam)
  • Regression
    • Example: Price of a car

Source: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow

11 of 33

Machine Learning - Supervised Learning

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Some of the most important supervised learning algorithms:
    • k-Nearest Neighbors (KNN)
    • Linear Regression
    • Logistic Regression
    • Support Vector Machines (SVMs)
    • Decision Trees and Random Forests
    • Naïve Bayes model
    • Neural networks

12 of 33

KNN

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • KNN, is the machine learning algorithm that uses proximity to compare one data point with a set of data it was trained on and make predictions.

13 of 33

KNN

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • K =3 we choose red
  • K =5 we choose blue

14 of 33

Decision Tree

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • A decision tree is a decision support recursive partitioning structure that uses a tree-like model of decisions and their possible consequences, including chance event outcomes, resource costs, and utility.

15 of 33

Decision Tree

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Internal node: input features
  • Leaf node: predictions

16 of 33

Decision Tree Overfitting

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • if the model becomes too complex, adjusts to every point, it overfits the training data, and struggles to generalize to new data. This is overfitting, where the model performs well on training but poorly on testing.
  • Decision trees tend to overfit because they try to perfectly classify the training data, which makes them highly sensitive to noise and minor fluctuations in the data. 

​

17 of 33

Random Forest

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Random Forest is a machine learning algorithm that uses multiple decision trees to make better predictions, and their results are combined by voting

18 of 33

Naïve Bayes

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • uses probability to predict which category a data point belongs to
  • assumes that the features are conditionally independent, given the target class.

19 of 33

Naïve Bayes

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

20 of 33

Logistic Regression

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

21 of 33

What is Unsupervised Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Unsupervised learning is a type of machine learning where the model is trained on an unlabeled dataset. Here, only the input data (features) are provided, and the model must find structure or relationships within this data.
  • This type of learning is commonly used in tasks such as clustering (grouping similar instances together) and dimensionality reduction (simplifying the inputs without losing too much information).
    • Examples include customer segmentation or reducing a high-dimensional dataset for visualization.

22 of 33

Machine Learning -Unsupervised Learning

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

Some of the most important unsupervised learning algorithms:

  • K-Means (Clustering Algorithm)
    • A popular clustering algorithm that partitions a dataset into 'k' distinct, non-overlapping clusters based on distance metrics, aiming to minimize the within-cluster distances and maximize the between-cluster distances.
  • Principal Component Analysis (PCA)
    • A dimensionality reduction technique that transforms a large set of correlated variables into a smaller set of uncorrelated variables, called principal components, which retain most of the original data's variation.

23 of 33

K-means

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

Partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or cluster centroid)

24 of 33

What is Reinforcement Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Reinforcement learning is a type of machine learning where an agent learns to make decisions by performing actions in an environment. The agent receives feedback in the form of rewards or penalties and aims to maximize the total reward over time.
  • This type of learning is often used in tasks that involve sequential decision-making, such as playing a game or controlling a robot. The goal is to learn a policy, which is a mapping from states to actions that maximizes the expected cumulative reward.

Source: Agent learning to walk from Unity Game Engine

25 of 33

What is Reinforcement Learning?

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

Source: Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow

2836584

26 of 33

Q learning

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

Source: https://www.geeksforgeeks.org/q-learning-in-python/

27 of 33

Q learning

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • The learning rate determines to what extent newly acquired information overrides old information.

​

  • A factor of 0 makes the agent exclusively exploiting prior knowledge, ignore new information entirely

​

  • A lower learning rate makes your estimates change slowly which can help stabilize learning in noisy environments

but may slow convergence

​

  • A factor of 1 makes the agent consider only the most recent

​

  • A higher learning rate means you put more weight on the latest observed reward means you put more weight on the

latest observed reward, can be very unstable if rewards are noisy.

28 of 33

Some Applications

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

  • Facial Recognition
  • Recommender System
  • Email spam filtering
  • Voice to text
  • Fraud detection
  • Self driving cars
  • Medical diagnosis
  • Image Recognition

29 of 33

Recommender System

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

30 of 33

Recommender System

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

Content-based filtering: Recommending content based on the content you like.

​

Collaborative filtering:

Recommending content based on what other people with similar preferences have liked.

31 of 33

Neural Network

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

32 of 33

RNN

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications

33 of 33

Thank you

REU Site: Applying Data Science on Energy-efficient Cluster Systems and Applications