Machine Learning

Prof. Seungtaek Choi

Do You Know Nano Banana? (1)

• Google’s image generation AI (namely Gemini 2.5 Flash Image)

Prompt: Show me a man who is 33 years old, actually a professor at Hankuk University of Foreign Studies, and give a lecture about Linear Regression.

Do You Know Nano Banana? (2)

• https://www.instagram.com/p/DOCZxPtCbdJ/

Nano Banana + Kling

Do You Know Nano Banana? (3)

• https://x.com/GoogleAIStudio/status/1964024315638403231

Last Time

• Course Overview
• Introduction to AI

Today

• Announcement: 1^st Assignment!
• Git/GitHub Basics
• Introduction to ML
• Definition
• Supervised Learning vs. Unsupervised Learning
• Classification, Regression, Dimension Reduction, Clustering, Model Selection
• Linear Regression
• Gradient Descent Algorithm

1^st Assignment!

• Assignment #1: Write your introduction and submit PR
• Deadline: 11:59 PM at Sep 17^th (1 week)
• Practice your Git and GitHub
• Feel free to react each other’s introduction (that’s also review!)
• Repository: https://github.com/HUFS-LAI-Seungtaek/HUFS-LAI-ML-2025-2
• Follow the instructions in https://github.com/HUFS-LAI-Seungtaek/HUFS-LAI-ML-2025-2/tree/main/assignments/assignment1

​

​

Git/GitHub Basics�(Or, How to Submit Assignment)

What is Git?

• Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.

What is GitHub?

• GitHub is a cloud-based platform built on the "Git" version control system that provides tools for developers to store, manage, share, and collaborate on code and other files.

Git & GitHub?

• git: version control & code management (local)
• github: code storage (cloud)
• push: upload code to github

push

Components of Git: Repository

• Repository is a version-controlled project space that stores your files, branches, and full change history.

Components of Git: Branch

• Branch is an independent line of development – a named timeline of commits within a repo.

Components of Git: Commit

• Commit is a saved snapshot of changes with a message, author, and timestamp.

Git Branching Structure (Dev)

src: https://docs.symbol.dev/handbook/git-branching.html

branch

branch

branch

branch

branch

commit

Git Branching Structure (Ours)

main

main

HUFS-LAI-Seungtaek/HUFS-LAI-OOP-2025-2:main

hufs-student-2/HUFS-LAI-OOP-2025-2:main

main+1

assignment1.py

main+2

assignment1.py

main+2

main+1

main

hufs-student-1/HUFS-LAI-OOP-2025-2:main

main+1

assignment1.py

Git Workflow

Git Workflow

• Repository structure:
• upstream (professor:main): original repo
• origin (student:main): your repo
• local: your computer

Git Workflow: fork

Git Workflow: clone

Git Workflow: add & commit

Git Workflow: add & commit

Git Workflow: add & commit

Git Workflow: add & commit

Git Workflow: add & commit

Git Workflow: push

Git Workflow: PR & merge

Git Workflow: branch

• For your assignment, no need to use `branch`
• professor:main (remote) 🡪 fork
• student:main (remote) 🡪 clone
• student:main (local) 🡪 commit
• student:main+1 (local) 🡪 commit
• student:main+2 (local) 🡪 push
• student:main+2 (remote) 🡪 PR
• professor:main+1 (remote)
• It’s not about remote vs. local.
• It’s about w/ permission vs. w/o permission.

Git Workflow: branch

• In your team’s repo, if you are not allowed to push `main` branch, …
• student:main (remote) 🡪 clone
• student:main (local) 🡪 checkout (In this case, $ git checkout –b feature)
• student:feature (local) 🡪 commit
• student:feature+1 (local) 🡪 commit
• student:feature+2 (local) 🡪 push
• student:feature+2 (remote) 🡪 PR
• student:main+1 (remote)

GitHub Web Shortcuts �(Or, How to Submit Assignment #1)

Fork repository

Fork repository

Repo is copied under your account.

Add a file

Add a file

Add a file

members/{학생이름}.md

Example: members/seungtaek.md

Example: members/yeachan.md

Example: members/gildong.md

Don’t include {}�Don’t use uppercase

Don’t use Korean

Introduce yourself

You can see actual “code” from

https://github.com/HUFS-LAI-Seungtaek/HUFS-LAI-OOP-2025-2/blob/main/members/seungtaek.md?plain=1

Feel free to introduce yourself more! (but, PLEASE FOLLOW THE OVERAL FORMAT!)

Commit the change (your file)

Back to “your” repo

Submit PR to lecture repository

Submit PR to lecture repository

Submit PR to lecture repository

Please follow the format �n-th Assignment by {학번} ({Full name})

�It’s important to check your submission status.

​

Do not use ‘ or “

Please use ` (look at the ~)

You can see the preview.

Introduction to ML

What is Machine Learning?

1. What is machine?
2. What is learning?
1. H. Simon: Any process by which a system improves its performance
2. H. Minsky: Learning is making useful changes in our minds
3. R. Michalsky: Learning is constructing or modifying representations of what is being experienced
4. L. Valiant: Learning is the process of knowledge acquisition in the absence of explicit programming

What is Machine Learning?

• Machine Learning
• “[the] field of study that gives computers the ability to learn without being explicitly programmed.” – Arthur L. Samuel (1959)
• “A learning machine, broadly defined, is any device whose actions are influenced by past experiences.” – Nils J. Nilsson (1965)
• “Pattern recognition – the act of taking in raw data and taking an action based on the ‘category’ of the pattern.” – Duda & Hart (1973)
• “The study and computer modeling of learning processes in their multiple manifestations constitutes the subject matter of machine learning” – Carbonell, Michalski & Mitchell (1983)
• “A computer program is said to learn from experience E … if its performance at tasks in T, as measured by P, improves with experience E.” – Tom M. Mitchell (1997)
• “The goal of machine learning is to program computers to use example data or past experience to solve a given problem.” – Ethem Alpaydin (2004)
• “[We] define machine learning as a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data.” – Kevin P. Murphy (2012)
• “ML models use big data to learn and improve predictability and performance automatically … without being programmed … by humans.” – OECD (2020s)

​

​

​

What is Machine Learning?

• Machine Learning
• “[the] field of study that gives computers the ability to learn without being explicitly programmed.” – Arthur L. Samuel (1959)
• “A learning machine, broadly defined, is any device whose actions are influenced by past experiences.” – Nils J. Nilsson (1965)
• “Pattern recognition – the act of taking in raw data and taking an action based on the ‘category’ of the pattern.” – Duda & Hart (1973)
• “The study and computer modeling of learning processes in their multiple manifestations constitutes the subject matter of machine learning” – Carbonell, Michalski & Mitchell (1983)
• “A computer program is said to learn from experience E … if its performance at tasks in T, as measured by P, improves with experience E.” – Tom M. Mitchell (1997)
• “The goal of machine learning is to program computers to use example data or past experience to solve a given problem.” – Ethem Alpaydin (2004)
• “[We] define machine learning as a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data.” – Kevin P. Murphy (2012)
• “ML models use big data to learn and improve predictability and performance automatically … without being programmed … by humans.” – OECD (2020s)

​

​

​

What is Machine Learning?

https://velog.io/@ddizzang/LG-Aimers-5%EA%B8%B0-module2-ML%EA%B0%9C%EB%A1%A0

Why study machine learning?

• Easier to build a learning system than to hand-code a working program!
• Robot that learns a map of environment by exploring
• Programs that learn to play games by playing against themselves
• Improving on existing programs
• Instruction scheduling and register allocation in compilers
• Combinatorial optimization problems
• Discover knowledge and patterns in highly dimensional, complex data
• Sky surveys
• Sequence analysis in bioinformatics
• Social network analysis
• Ecosystem analysis

Very brief history

• Studied ever since computers were invented (e.g., Samuel’s checkers player)
• Very active in 1960s (neural networks)
• Died down in the 1970s
• Revival in early 1980s (decision trees, backpropagation, temporal-difference learning) - coined as “machine learning”
• Exploded starting in the 1990s
• Now: very active research field, several yearly conferences (e.g., ICML, NeurIPS), major journals (e.g., Machine Learning, Journal of Machine Learning Research)
• The time is right to study in the field!
• Lots of recent progress in algorithms and theory
• Flood of data to be analyzed
• Computational power is available
• Growing demand for industrial applications

Very brief history

Ref: https://arxiv.org/abs/2109.01517

What are good machine learning tasks?

• There is no human expert
• E.g., DNA analysis, video recommendation
• Humans can perform the task but cannot explain how
• E.g., character recognition
• Desired function changes frequently
• E.g., predicting stock prices based on recent trading data
• Each user needs a customized function
• E.g., news filtering

Important application areas…

• Bioinformatics: sequence alignment, analyzing microarray data, information integration, …
• Computer vision: object recognition, tracking, segmentation, active vision, …
• Robotics: state estimation, map building, decision making
• Graphics: building realistic simulations
• Speech: recognition, speaker identification
• Financial analysis: option pricing, portfolio allocation
• E-commerce: automated trading agents, data mining, spam, …
• Medicine: diagnosis, treatment, drug design, …
• Computer games: building adaptive opponents
• Multimedia: retrieval across diverse databases

Supervised Learning

vs.

Unsupervised Learning

Supervised Learning

Supervised Learning Example 0: Linear function

Supervised Learning Example 0: Linear function

Supervised Learning Example 1: Housing price prediction

• You can practice on Kaggle:
• House Prices - Advanced Regression Techniques | Kaggle

​

Supervised Learning Example 2: Breast cancer (malignant, benign)

Supervised Learning Example 2: Breast cancer (malignant, benign)

Supervised Learning Example 3: Spam Detection

Unsupervised Learning

• Training experience: unlabeled data
• What to learn: interesting associations in the data
• E.g., image segmentation, clustering
• Often there is no single correct answer

Unsupervised Learning Example 1: Gene Clustering

• Training experience: unlabeled data
• What to learn: interesting associations in the data
• E.g., image segmentation, clustering
• Often there is no single correct answer

Unsupervised Learning Example 2: Customer Segmentation

src: https://medium.com/analytics-vidhya/customer-segmentation-for-differentiated-targeting-in-marketing-using-clustering-analysis-3ed0b883c18b

https://www.kaggle.com/code/emineyetm/customer-segmentation-with-unsupervised-learning

Supervised Learning vs. Unsupervised Learning

Reinforcement Learning

• Problems involving an agent interacting with an environment which provides numeric reward signals
• Goal: Learn how to take actions in order to maximize reward

src: https://www.youtube.com/watch?v=qv6UVOQ0F44

Machine Learning – Taxonomy of Problems

• Classification
• Regression
• Density Estimation
• Dimension Reduction
• Clustering
• Model Selection

Classification

Regression

Density Estimation

Dimension Reduction

• In dimension reduction, one attempts to learn a low-dimensional manifold to represent complex data.
• e.g. PCA (Principal Component Analysis), ICA (Independent Component Analysis)

Clustering

• Clustering refers to techniques to segmenting data into coherent “clusters”
• e.g. k-means, Mixtures-of-Gaussians, mean-shift

Model Selection

• “Given a choice of two models, which one is more appropriate to the data?”
• “How big should the model be?”
• e.g. A model with more parameters will fit the data better, but it could also overfit the data.

Thus far…

Supervised vs. unsupervised learning

Different types of machine learning problem

• Classification
• Regression
• Density Estimation
• Dimension Reduction
• Clustering
• Model Selection

Supervised Learning /

Regression

Supervised Learning Problem

Housing price prediction (again)

Housing price prediction (again)

Housing price prediction (again)

Housing price prediction (again)

Gradient Descent Algorithm

Gradient Descent Algorithm �for Linear Regression

Linear Regression �with Multiple Features

Linear Regression with Multiple Variables�- Gradient Descent in Practice

Linear Regression with Multiple Variables�- Normal Equation

Linear Regression with Multiple Variables�- Features and Polynomial Regression

Next

• Logistic Regression
• Classification