3 of 27

NoSQL

(mongoDB)

spreadsheetscsv files

relational databases

small amount of data

big amount of data

Big Data

Data evolution and the rise of ML

This slide sets the historical context for why Machine Learning emerged as a discipline.

With the advent of computers, data analysis became faster and more scalable. For small to medium datasets, spreadsheets (Excel, CSV files) and statistical software like R were sufficient: they allow storing, filtering, and analysing tabular data intuitively.

As data volumes grew, relational databases (e.g., MySQL, PostgreSQL) became essential. These use SQL (Structured Query Language) to store, query, and manipulate data in a structured, schema-defined way. SQL databases excel at managing structured data with well-defined relationships between entities.

The term "Big Data" emerged in the 2000s to describe datasets too large, fast, or heterogeneous for traditional tools. Tech giants like Google, Amazon, Facebook, and Twitter began accumulating massive, often unstructured data: user actions, click histories, social interactions, purchasing patterns. Relational databases were not designed for this kind of data. This gave rise to NoSQL systems like MongoDB, which store data in flexible, document-oriented formats (BSON, similar to JSON), without requiring a fixed schema. MongoDB organises data into collections and documents, enabling scalability across distributed servers.

A biological parallel: in February 2001, the first draft of the human genome was published. With approximately 3.1 billion base pairs per genome, and with thousands of genomes now being sequenced routinely, this represents genuine Big Data. No human analyst can inspect thousands of genomes manually to find patterns — we need automated, scalable approaches. This is precisely where Machine Learning became indispensable: it allows us to extract patterns and make predictions from massive datasets that exceed human cognitive capacity.

Key drivers of the ML revolution:

- Explosive growth in data (from businesses and science alike)

- Improvements in CPUs and especially GPUs (Graphics Processing Units), enabling fast parallel computation

- Development of open-source ML frameworks and libraries

Key point to make to students: ML did not emerge in a vacuum. It is the natural response to a data landscape that outgrew traditional analytical tools.

MongoDB

MongoDB is a database — a system for storing and retrieving data — but it works differently from traditional databases like MySQL or PostgreSQL.

Here's the key difference in simple terms:

Traditional databases store data in tables (like spreadsheets), with rows and columns. Everything must fit a strict structure.

MongoDB stores data as documents — think of them like JSON objects or flexible "cards" that can hold any kind of information, in any shape.

For example, instead of a table row like:

Idnameage 1 Anna 30

MongoDB stores it as:

json{ "id": 1, "name": "Anna", "age": 30, "hobbies": ["reading", "hiking"] }

Notice you can add extra fields (like hobbies) without affecting other documents — no rigid structure required.

Why use MongoDB?

Flexible: No need to define a fixed schema upfront — great when data evolves over time.

Scalable: Handles very large amounts of data well.

Developer-friendly: Data looks like objects programmers already work with in code.

Fast for certain queries: Especially when reading entire documents at once.

When is it a good fit?

Apps with rapidly changing data structures
Content management systems, user profiles, catalogs
Real-time apps (chat, analytics)

When might it not be ideal?

When you have highly relational data with many interconnected tables (a traditional SQL database may be better)
When strict data consistency is critical (e.g., banking transactions)

In short: MongoDB is a flexible, document-based database that trades rigid structure for speed and adaptability.

4 of 27

Me: “Could you give a definition of ML in 15 words?”

ChatGPT: “A field of study enabling computers to learn from data and make predictions autonomously”

Me: “What ML does (in 20 words)?”

ChatGPT: “ML analyzes data patterns to make predictions, automate tasks, improve decision-making, and enable systems to learn without explicit programming"

What is Machine Learning? — A ChatGPT-assisted definition

This slide uses a conversational, interactive format to define ML — deliberately inviting students to engage critically with AI-generated content.

You asked ChatGPT two questions:

- "Give a definition of ML in 15 words" → "A field of study enabling computers to learn from data and make predictions autonomously."

- "What ML does (in 20 words)?" → "ML analyzes data patterns to make predictions, automate tasks, improve decision-making, and enable systems to learn without explicit programming."

These definitions capture the essence well. Key concepts to unpack:

- "Learn from data": the model is not programmed with explicit rules; it infers rules from examples.

- "Without explicit programming": this distinguishes ML from traditional rule-based programming, where every decision path must be hand-coded.

- "Make predictions autonomously": once trained, the model can generalise to new, unseen data.

Useful framing for students: In traditional programming, a human writes rules → the computer applies them to data → produces output. In ML, the process is inverted: data + desired outputs → the computer learns the rules.

Transition question for the class: "If ML learns from data, HOW does it actually learn? What mechanisms does it use?" This naturally leads to the next slide on ML types.

5 of 27

The three different types of (traditional) ML

Supervised learning

Unsupervised learning

Reinforcement learning

The three types of traditional ML — Overview

This slide introduces the three major paradigms of Machine Learning. Emphasise that all three paradigms share the same fundamental goal: predicting results based on incoming data. They differ in what information is available during training and what kind of output is expected.

The three categories are:

1. Supervised Learning — learns from labelled examples (input-output pairs)

2. Unsupervised Learning — finds structure in unlabelled data

3. Reinforcement Learning — learns through trial and error, guided by rewards

These three categories are not mutually exclusive in practice — hybrid approaches exist — but they provide a useful conceptual framework.

Pedagogical note: at this point, you might ask students to intuitively predict what "supervised" vs "unsupervised" might mean before revealing the definitions. This activates prior knowledge and sets up meaningful expectations.

6 of 27

Supervised learning

Unsupervised learning

Reinforcement learning

The three different types of (traditional) ML

> Labelled data

> Direct feedback

> Predict

outcome/future

[SLIDE: Supervised Learning — Key characteristics]

Supervised learning is the most widely used paradigm in ML and the one students will work with most in this course.

Definition: A supervised learning algorithm learns a mapping function from input features to output labels, based on a labelled training dataset — a set of examples where both the input and the correct output are known.

The term "supervised" is analogous to learning with a teacher: the algorithm receives feedback at every step, comparing its predictions to the known correct answers. This feedback is used to adjust the model’s parameters iteratively.

Three core properties:

- Labelled data: each training example has a known target value (a class label or a numerical outcome)

- Direct feedback: the algorithm can measure how wrong it is (error/loss) and use this to improve

- Predict outcome/future: once trained, the model generalises to new data it has never seen

Important concept — the training-test distinction: a model is trained on a subset of the data and evaluated on a held-out test set. This mimics the real-world scenario where we use a model on new, unseen examples.

Intuitive analogy: supervised learning is like studying for an exam with answer keys available. You know which answers are correct, and you learn from your mistakes. At test time, you face new questions without the answer key.

7 of 27

Supervised learning

Unsupervised learning

Reinforcement learning

The three different types of (traditional) ML

> Email spam filtering

> Protein secondary

structure prediction

> Free energy of

folding after mutating

a residue in a protein

Supervised Learning — Biological and practical examples

This slide grounds supervised learning in concrete biological and everyday contexts.

Email spam filtering (classic ML example): a model is trained on a labelled corpus of emails (spam / not spam). It learns patterns (certain words, sender addresses, link density) associated with each class. When a new email arrives, it assigns a probability of belonging to each class. This is a binary classification task.

Protein secondary structure prediction: this is a key bioinformatics application. Each residue in a protein sequence is predicted as belonging to one of three structural classes: Helix (H), Strand (E), or Loop/Coil (C). The ground truth labels come from crystallographic data deposited in the Protein Data Bank (PDB). The model is trained on known protein structures and then used to predict structure from sequence alone.

Other important biological examples of supervised learning:

- Disease diagnosis from medical imaging (e.g., classifying tissue as benign or malignant from histology images)

- Protein family classification from amino acid sequences

- Species identification from DNA barcodes or morphological features

- Cell type classification in single-cell RNA-seq data (scRNA-seq), where cells are labelled based on known marker genes

- Microbial classification from 16S rRNA metagenomic sequences

Key concept to emphasise: the quality and representativeness of the training labels directly determines the quality of the model. Biased or poorly annotated training data will produce biased models — a critical issue in biomedical applications.

8 of 27

Supervised

Unsupervised

Reinforcement

The three different types of (traditional) ML

Classification for predicting class labels

Supervised Learning — Classification

Classification is the most common supervised learning task. Here, the output (label) is a discrete, categorical variable.

Formal definition: given a set of input features x, a classifier learns a function f(x) → y, where y belongs to a finite set of predefined categories (classes). The classes are unordered — there is no intrinsic ranking between them.

Binary classification: two possible output classes (e.g., spam/not-spam, pathogenic/benign variant, tumour/normal tissue). The model learns a decision boundary that separates the two classes in feature space.

Multiclass classification: three or more possible output classes (e.g., secondary structure prediction: Helix, Strand, Coil; or cell type classification in scRNA-seq with dozens of cell types).

Important concept — the decision boundary: the classifier does not simply memorise examples; it learns a generalised rule (the decision boundary) that can correctly assign class membership to new, unseen data points. The best boundary minimises misclassification.

In the Iris dataset example (slide 12), the features are petal and sepal measurements and the task is to classify each flower into one of three species. This dataset is a canonical teaching tool because it is small, well-understood, and illustrates the principles of multiclass classification clearly.

9 of 27

Classifying a new data point

Classifying a new data point — Decision boundaries

This slide illustrates the concept of a decision boundary with a two-dimensional, binary classification example.

The figure shows 30 training points: 15 labelled Class A and 15 labelled Class B. Each point is described by two features, x1 and x2 (e.g., two gene expression measurements, two morphological traits).

The classifier’s job is to learn a boundary (the dashed line) that best separates the two classes. Once this boundary is learned, any new, unlabelled point can be classified by determining which side of the boundary it falls on.

Key insight: there are infinitely many possible decision boundaries that could separate the classes. What makes a boundary "good"? It should minimise the number of training examples misclassified and, more importantly, generalise well to new data (i.e., avoid overfitting).

Extension to multiple classes: the same principle applies to multiclass settings. For example, handwritten digit recognition (0–9) requires 10 classes. The model must be trained on examples of each digit and cannot classify digits it has never seen in training.

Practical implication: if a feature not present in training (e.g., eye colour in a model trained on clinical measurements) is provided at test time, the model cannot use it — and conversely, if a critical feature is missing at test time, performance will degrade.

Pause and reflect for students: "Can you think of a biological classification problem where the decision boundary might not be a straight line? What would that imply about the data?" (This can foreshadow non-linear classifiers and kernel methods.)

10 of 27

id	gender	age	heigh (cm)	weight (kg)	Max blood pressure (mmHg)	responsiveness to treatment (label)
1	F	27	174	61	128	yes
2	F	32	169	58	123	yes
3	M	26	177	68	158	no
4	M	38	184	88	118	yes
5	F	77	162	51	148	no
…	…	…	…	…	…	…

Regression/Classification example — Medical dataset

This dataset illustrates supervised learning applied to a clinical context — predicting patient responsiveness to a treatment.

Features: gender, age, height (cm), weight (kg), maximum blood pressure (mmHg)

Label: responsiveness to treatment (yes/no — note this is actually a binary classification label, not a continuous value)

This example is pedagogically important for several reasons:

1. Anonymisation: patient IDs are used instead of names — a reminder of data privacy requirements in biomedicine (GDPR, ethical data handling)

2. Classification vs regression: although the dataset looks similar to the salary example, the label "responsiveness to treatment" is categorical (yes/no), making this a classification problem. Ask students to spot this difference.

3. Generalisation from training examples: the model can predict responsiveness for a new patient whose feature values have never been seen exactly — it generalises the learned patterns. However, it cannot predict based on features not in the training set (e.g., eye colour would be useless here).

4. Missing features: if a key feature like blood pressure is missing for a new patient, prediction quality will likely drop.

5. Clinical relevance: personalised medicine increasingly relies on ML models trained on patient data to predict treatment outcomes, stratify risk, and tailor interventions. This example directly reflects that application area.

Discussion prompt: "What ethical issues might arise when using a model like this in a real clinical setting? What kinds of biases could creep into the training data?"

11 of 27

The Iris dataset — A canonical ML benchmark

The Iris dataset is one of the most famous and widely used datasets in ML education, introduced by statistician Ronald Fisher in 1936. It is an excellent starting point for learning classification.

Dataset summary:

- 150 examples (flower samples), 50 per species

- Three classes: Iris setosa, Iris versicolor, Iris virginica

- Four features: sepal length, sepal width, petal length, petal width (all in centimetres)

Why is this dataset so valuable for teaching?

- It is small enough to visualise and inspect manually

- It illustrates multiclass classification (3 classes)

- Some class pairs are linearly separable (setosa vs the others), while others require a non-linear boundary (versicolor vs virginica) — demonstrating that real data can be complex

- It has no missing values and is well-balanced, making it an ideal clean introduction

Students will likely encounter the Iris dataset in their first hands-on ML exercises. They should become comfortable with its structure: rows = examples/observations, columns = features, final column = class label.

Bridge to practice: "When you load this dataset in Python using scikit-learn, what does the data matrix look like? How many rows and columns? What does each row represent?"

12 of 27

Supervised

Unsupervised

Reinforcement

The three different types of (traditional) ML

Regression for predicting continuous outcomes

Supervised Learning — Regression

Having covered classification (predicting categorical labels), we now introduce regression — the second major type of supervised learning.

Regression predicts a continuous numerical output (the target variable) from one or more input features. Unlike classification, where the output is a class membership, in regression the output is a real-valued number on a continuous scale.

Examples of regression tasks in biology and biomedicine:

- Predicting the free energy change (ΔΔG) of protein folding after a point mutation

- Estimating gene expression levels from genomic features

- Predicting patient age from DNA methylation patterns (epigenetic clocks)

- Modelling drug dose-response relationships

Three common regression types:

Linear regression: models a straight-line relationship between one input (x) and the output (y). Formula: y = mx + b. Example: predicting height from age in growing children.

Multiple linear regression: extends to multiple input features. Formula: y = b₀ + b₁x₁ + b₂x₂ + ... + bₙxₙ. Example: predicting protein stability from multiple biophysical parameters.

Polynomial regression: captures non-linear relationships by fitting a polynomial curve. Example: modelling population growth or enzyme kinetics.

Key distinction from classification: regression output is on a continuous scale; there are no discrete categories. The loss function used to train the model (typically mean squared error, MSE) penalises numerical distance from the true value, rather than binary correct/incorrect classification.

However, we can have categorical features, but these need encoding.

Examples:

Genotype (AA/Aa/aa) → ordinal or one-hot depending on the biological assumption (additive vs. dominant model)
Tissue type or cell line → nominal → one-hot
Presence/absence of a mutation → binary → 0/1

Indeed, in supervised regression algorithms, both the features and the target must ultimately be numeric — but the reasons and nuances differ:

Target variable: must always be a continuous numeric value. This is definitional: regression predicts a quantity (e.g., gene expression level, protein stability score, patient age). If your target is categorical, that's a classification problem.

Features: must be numeric at the time the algorithm processes the data, but they don't have to start as numbers. Categorical features need to be encoded first:

Ordinal categories (e.g., "low/medium/high") → integer encoding (0, 1, 2) preserving order
Nominal categories (e.g., tissue type, amino acid identity) → one-hot encoding (a binary column per category), to avoid implying a false order
Binary categories (e.g., yes/no, mutant/wildtype) → 0/1 directly

A few practical notes:

Missing values also need handling before training (imputation or removal), since most algorithms can't process NaN
Some tree-based algorithms (like certain implementations of Random Forest or XGBoost) can handle categorical features natively, but scikit-learn's implementations still require numeric input
Text features require vectorisation (TF-IDF, embeddings, etc.) before they can enter a regression model

So the mental model is: raw data can be mixed, but the matrix that enters the algorithm must be all-numeric — encoding is the bridge. This is exactly the kind of preprocessing step that belongs before your train/test split in the teaching workflow you use.

13 of 27

Predicting the target variable of new data

Example of linear regression

Regression — Linear regression example

This slide illustrates simple linear regression with a single input feature predicting a continuous outcome.

In linear regression, the goal is to find a straight line (y = mx + b) that best fits the relationship between the input feature (x) and the continuous target variable (y). "Best fit" is typically defined as minimising the mean squared error (MSE) — the average squared distance between each observed data point and the line’s predicted value.

Example from the slide: predicting SAT math scores from study time. Given labelled training data (study hours, SAT score pairs), the model learns the slope and intercept that minimise prediction error.

Biological example: predicting protein folding free energy change after mutation. Given a training set of experimentally measured ΔΔG values with associated molecular features, a regression model can predict stability changes for new mutations.

Key difference from classification: in regression, y is a real number; in classification, y is a category. The visualisation changes accordingly: regression produces a fitted curve or surface; classification produces a decision boundary.

Residuals: the vertical distance between each data point and the regression line is called a residual. Minimising the sum of squared residuals is the objective of ordinary least squares (OLS) regression.

Visual intuition to share: "If you were to draw a line through a scatter plot by hand to best represent the trend, you would be doing something very similar to what linear regression does — but the algorithm finds the mathematically optimal line."

14 of 27

Regression example — Salary prediction dataset

This dataset exemplifies a regression task, where the target variable (Salary) is continuous rather than categorical.

Features in this dataset include: Position, Experience (years), Skill, Country, City. The label is Salary (in $). An ML regression model trained on this data learns the relationship between these input features and the numerical salary outcome, enabling it to predict the salary of a new employee given their profile.

Key concepts for this slide:

- Features (also called predictors or independent variables): the input columns used to make predictions

- Label (also called target or dependent variable): the output to be predicted — in this case, a continuous number

- Prediction at test time: given a new data point with known features (but unknown salary), the model produces a numerical estimate

This simple dataset helps students understand what a tabular dataset for regression looks like before introducing more abstract notation. Emphasise that the model does not simply look up exact matches in the training data — it generalises the relationship to new, unseen combinations of feature values.

Ask students: "If you were building this model, which features do you think would be most predictive of salary? Why?" This encourages feature relevance thinking before introducing formal feature importance methods.

15 of 27

Supervised learning - label learning

Supervised learning — Summary diagram

This slide summarises the supervised learning workflow visually. Use it as a consolidation moment before moving on.

The key steps in supervised learning:

1. Collect labelled training data (features + known labels)

2. Train the model: the algorithm learns a mapping from features to labels

3. Evaluate: test the model on held-out data to measure generalisation

4. Deploy: use the model to predict labels on new, unseen data

"Label learning" is the informal but intuitive way to describe supervised learning: you teach the model by showing it labelled examples repeatedly until it can correctly predict labels on its own.

Reflective question for students: "How do you think training data and labels look like in practice for the examples we have seen so far — protein secondary structure prediction, treatment response, the Iris dataset?"

This is a good moment to pause and check understanding before introducing unsupervised learning. You might ask students to sketch what they think the training pipeline looks like for one of the biological examples discussed.

16 of 27

Supervised learning

Unsupervised learning

Reinforcement learning

> Labelled data

> Direct feedback

> Predict

outcome/future

The three different types of (traditional) ML

> No labels/targets

> No feedback

> Find hidden

structure in data

Unsupervised Learning — Key characteristics

Transition from supervised to unsupervised learning. The key contrast is the absence of labels.

In supervised learning, we have a dataset with both input features and known output labels. The algorithm learns to map inputs to outputs. In unsupervised learning, we only have input features — no labels, no correct answers, and no direct performance signal.

The goal shifts from prediction (what is the correct label?) to discovery (what structure or patterns exist in this data?).

Three core properties of unsupervised learning:

- No labels/targets: the dataset has features but no pre-assigned categories or values to predict

- No feedback: we cannot directly measure whether the algorithm is "correct" — there is no ground truth to compare against

- Find hidden structure: the algorithm must discover groupings, patterns, or compressed representations on its own

Analogy for students: imagine receiving a large pile of mixed documents with no titles or categories. Unsupervised learning is like trying to sort them into meaningful piles based on content similarity — without being told how many piles to make or what they should contain.

In bioinformatics, this scenario is extremely common: gene expression datasets, protein sequence collections, patient omics profiles often arrive without neat labels. Unsupervised learning helps reveal natural groupings, subtypes, and structure in these data.

17 of 27

Unsupervised learning – Clustering

Unsupervised Learning — Clustering

Clustering is the most intuitive and widely used form of unsupervised learning. Its goal is to partition a dataset into groups (clusters) such that points within the same cluster are more similar to each other than to points in other clusters.

Illustrated example: the figure shows data points described by two features (x1 and x2). Without any class labels, the algorithm identifies three natural groups (clusters A, B, C) based on the proximity of points in feature space. Points close together are more likely to share underlying characteristics.

How similarity is measured: clustering algorithms use distance metrics (e.g., Euclidean distance, cosine similarity, correlation) to quantify how similar two data points are. Denser regions in feature space tend to form clusters.

Common clustering algorithms:

- K-means: assigns each point to the nearest of k centroids; iterates until stable. Requires specifying k in advance.

- Hierarchical clustering: builds a tree of nested clusters (dendrogram); does not require specifying k. Common in genomics.

- DBSCAN: density-based; identifies clusters as dense regions separated by sparse areas; can detect noise/outliers.

Biological applications:

- Gene expression clustering: grouping samples or genes with similar expression patterns (e.g., identifying tumour subtypes from RNA-seq data)

- Metagenomics: clustering microbial sequences into operational taxonomic units (OTUs)

- Patient stratification: clustering patients based on multi-omics data to identify disease subtypes

Key insight: clustering does not produce a "right" answer — different algorithms, distance metrics, and parameter choices can yield different groupings. Interpretation requires domain knowledge.

18 of 27

Sample ID	Gene A	Gene B	Gene C	Gene D	Gene E	Tissue Type (Hidden)
1	8.2	3.1	5.5	7.8	6.0	Brain
2	2.3	9.0	1.2	4.5	7.5	Liver
3	6.5	2.9	8.3	3.4	5.1	Brain
4	1.1	8.5	2.7	5.6	9.2	Liver
5	7.8	3.4	6.1	8.9	4.3	Brain
6	3.0	7.9	2.0	4.2	8.7	Liver
7	9.5	2.0	7.1	5.7	3.8	Brain
8	4.2	8.8	1.5	6.9	7.3	Liver

Example Dataset: Gene Expression in Tissue Samples

Unsupervised Learning — Gene expression dataset example

This slide presents a concrete biological dataset to illustrate unsupervised learning in a genomics context.

The dataset contains gene expression levels (Genes A–E) for 8 tissue samples. The "Tissue Type" column is shown for illustrative purposes — in a real unsupervised learning scenario, this information would NOT be available to the algorithm.

Goal: can an unsupervised algorithm discover that Brain and Liver samples form distinct clusters, based only on their gene expression profiles?

Approaches applicable to this dataset:

1. Clustering (K-means, Hierarchical Clustering, DBSCAN):

- Group samples with similar gene expression patterns

- In this example, Brain samples (high Gene A, Gene C, Gene D) and Liver samples (high Gene B, Gene E) should separate naturally

- Useful for discovering unknown tissue types or novel biological subtypes without prior labelling

2. Dimensionality Reduction (PCA, t-SNE, UMAP):

- With only 5 genes this dataset is already visualisable, but real scRNA-seq datasets contain thousands of genes

- PCA (Principal Component Analysis) finds linear combinations of genes that capture the maximum variance

- t-SNE and UMAP are non-linear methods better suited for preserving local structure in high-dimensional biological data

- These methods allow visualising thousands of genes in 2D or 3D to reveal clusters and trajectories

Real-world connection: single-cell RNA sequencing (scRNA-seq) generates expression data for tens of thousands of genes across thousands of individual cells. Unsupervised methods are essential for identifying cell types, developmental trajectories, and disease states in these massive datasets.

19 of 27

Supervised learning

Unsupervised learning

Reinforcement learning

> Labelled data

> Predict outcome

> Direct feedback

The three different types of (traditional) ML

> No labels/targets

> Make sense in data

> No feedback

Supervised vs Unsupervised — Side-by-side comparison

This slide directly contrasts the two paradigms to consolidate understanding before introducing reinforcement learning.

Supervised learning:

- Has labelled training data (input-output pairs)

- Directly measures performance: the model’s predictions can be compared to known correct answers

- Goal: learn a mapping that generalises from training labels to new data

- Examples: classification, regression

Unsupervised learning:

- No labels or target values available

- No direct performance measure: there is no "correct" answer to compare predictions against

- Goal: discover hidden structure, groupings, or compressed representations in the data

- Examples: clustering, dimensionality reduction

An important subtlety: in unsupervised learning, we cannot tell the machine it is right or wrong in the way we can in supervised learning. Instead, we evaluate the quality of discovered structure indirectly (e.g., silhouette score for clustering quality, or explained variance for PCA).

Why does this matter in biology? In many omics studies, labelled data is scarce, expensive, or simply unavailable. Unsupervised methods allow exploration and hypothesis generation from raw data — a critical first step before targeted supervised analysis.

Discussion question for students: "Can you think of a biological problem where you would start with unsupervised learning and then transition to supervised learning? What would drive that transition?" (Answer: e.g., first cluster patient samples unsupervised → manually annotate discovered clusters → train a classifier on those labels)

20 of 27

Unsupervised learning - Dimensionality reduction

Unsupervised Learning — Dimensionality reduction

Dimensionality reduction is the second major subfield of unsupervised learning. Its goal is to reduce the number of features in a dataset while retaining as much relevant information as possible.

Why is this necessary?

- High-dimensional data (many features) is difficult to visualise, computationally expensive to process, and prone to the "curse of dimensionality" (distances become less meaningful in very high dimensions)

- Genomics routinely produces datasets with thousands or millions of features (genes, SNPs, methylation sites)

- Noise and redundancy in high-dimensional data can degrade model performance

What dimensionality reduction does: transforms the original high-dimensional feature space into a lower-dimensional representation (a "subspace") that captures the most important variation in the data. If two features are highly correlated, they carry redundant information — dimensionality reduction can combine them into a single, more informative composite feature.

The Swiss Roll example (in the figure): a 3D dataset is compressed into 2D while preserving the neighbourhood structure. Points that were close in 3D remain close in 2D.

Common methods:

- PCA (Principal Component Analysis): linear method; finds axes of maximum variance. Widely used in genomics for population structure analysis and batch effect correction.

- t-SNE: non-linear; preserves local structure; excellent for visualisation (but not for downstream analysis)

- UMAP: non-linear; faster than t-SNE; better preserves global structure; increasingly standard in scRNA-seq pipelines

Biological applications:

- scRNA-seq: reducing tens of thousands of gene dimensions to 2D/3D for cell type visualisation (UMAP plots are ubiquitous in single-cell biology papers)

- Population genetics: PCA of SNP data to detect population structure

- Protein function prediction: compressing protein sequence or structure features for clustering and function annotation

Feature selection vs feature extraction: feature selection retains a subset of original features; feature extraction (e.g., PCA) creates new composite features.

21 of 27

Supervised learning

Unsupervised learning

Reinforcement learning

> Labelled data

> Direct feedback

> Predict

outcome/future

The three different types of (traditional) ML

> No labels/targets

> No feedback

> Find hidden

structure in data

> Decision process

> Reward system

> Learn series of

actions

Reinforcement Learning — Key characteristics

Reinforcement Learning (RL) is the third major paradigm of ML. It is conceptually the most different from supervised and unsupervised learning and the closest in spirit to how animals (including humans) learn through experience.

In supervised learning: we have labelled examples → the model learns from known correct answers

In unsupervised learning: we have unlabelled data → the model discovers hidden structure

In reinforcement learning: an agent interacts with an environment → it learns which actions to take to maximise cumulative reward

Three core properties of RL:

- Decision process: the agent makes a sequence of decisions over time, not a single prediction

- Reward system: the agent receives feedback in the form of rewards (positive) or penalties (negative) for its actions — but crucially, this feedback is a measure of outcome quality, not a ground truth label

- Learn series of actions: the agent learns a policy — a mapping from states to actions — that maximises long-term reward

Analogy: teaching a dog to fetch. You don’t hand it a manual of instructions. Instead, you reward desired behaviours and ignore or correct undesired ones. Over many trials, the dog learns which behaviours lead to rewards.

Key difference from classification: when classifying email as spam, the model makes a single prediction per input. In RL, actions have consequences that unfold over time, and the model must learn to plan ahead. A move that earns a small immediate reward might lead to a loss later.

RL is central to: game-playing AI (AlphaGo, AlphaZero), robotics, drug dosing optimisation, adaptive clinical trials, and protein design with feedback.

Input data in reinforcement learning

In reinforcement learning, the input data is fundamentally different from the tabular format used in supervised/unsupervised learning. There is no pre-collected dataset at all — instead, the agent learns through interaction with an environment.

At each step, the agent receives a state (the current situation), which can take very different forms depending on the application:

a vector of numerical values (e.g., position and velocity of a robot)
a grid or image (e.g., a screenshot in a video game like Atari)
a graph (e.g., a molecular structure)
a set of sensor readings

The agent then takes an action, and the environment returns:

the new state
a reward (a scalar signal — positive, negative, or zero)

So the "input" is not a static table but a continuous stream of (state, action, reward, next state) tuples generated dynamically during training. These tuples are sometimes stored in a replay buffer for reuse, but they are produced by the agent's own experience, not collected in advance.

22 of 27

Reinforcement learning

Reinforcement Learning — Conceptual diagram

This slide presents the canonical reinforcement learning loop. Let’s walk through each component:

Agent: the decision-making entity (e.g., the chess programme, a robotic arm, a drug dosing controller)

Environment: the external system the agent interacts with (e.g., the chess board, the physical world, a patient)

State (s): a description of the current situation of the environment that the agent observes (e.g., current board configuration, current patient health metrics)

Action (a): what the agent does in a given state (e.g., move a chess piece, administer a dose)

Reward (r): the feedback signal the agent receives after taking an action — positive for good outcomes, negative (penalty) for bad ones

Policy (π): the strategy the agent learns — a mapping from states to actions that maximises cumulative reward over time

The RL loop:

1. Agent observes the current state of the environment

2. Agent selects an action according to its current policy

3. Environment transitions to a new state

4. Agent receives a reward signal

5. Agent updates its policy based on the reward → repeat

Important nuance: in RL, the reward is not a ground truth label. It is a scalar feedback signal indicating how good or bad the outcome was. The agent must figure out which of its past actions caused the reward — the credit assignment problem.

RL differs from supervised learning in that the feedback (reward) can be delayed (e.g., you only know you won or lost a game at the very end) and is not instructive (it does not tell the agent what the correct action was, only that the outcome was good or bad).

Note:

Reinforcement learning = a general framework for learning through interaction and reward signals
Deep RL = RL where the value function or policy is represented by a neural network

Many practical robotics, operations research, and classic game AI applications still use non-neural RL methods perfectly effectively.

23 of 27

The Reinforcement learning loop

1. Agent observes the current state of the environment

2. Agent selects an action according to its current policy

3. Environment transitions to a new state

4. Agent receives a reward signal

5. Agent updates its policy based on the reward → repeat

Reinforcement Learning — Conceptual diagram

This slide presents the canonical reinforcement learning loop. Let’s walk through each component:

Agent: the decision-making entity (e.g., the chess programme, a robotic arm, a drug dosing controller)

Environment: the external system the agent interacts with (e.g., the chess board, the physical world, a patient)

State (s): a description of the current situation of the environment that the agent observes (e.g., current board configuration, current patient health metrics)

Action (a): what the agent does in a given state (e.g., move a chess piece, administer a dose)

Reward (r): the feedback signal the agent receives after taking an action — positive for good outcomes, negative (penalty) for bad ones

Policy (π): the strategy the agent learns — a mapping from states to actions that maximises cumulative reward over time

The RL loop:

1. Agent observes the current state of the environment

2. Agent selects an action according to its current policy

3. Environment transitions to a new state

4. Agent receives a reward signal

5. Agent updates its policy based on the reward → repeat

Note:

Reinforcement learning = a general framework for learning through interaction and reward signals
Deep RL = RL where the value function or policy is represented by a neural network

Many practical robotics, operations research, and classic game AI applications still use non-neural RL methods perfectly effectively.

24 of 27

Reinforcement learning

Trainer

+ room

Situation

Dog

Example:

The simplest analogy is training a dog.

You don't show the dog a table of examples. Instead:

the dog (the agent) is in a situation (state): you're standing in front of it holding a treat
it tries something (action): it sits down
it gets a signal (reward): you give it the treat and say "good dog!"
The trainer + the room represent the environment
Getting better at obeying over time is the policy (the learned strategy: "in state X, do action Y")

Over many repetitions, the dog learns which actions lead to rewards in which situations — without ever having seen a dataset.

In reinforcement learning, the agent (the "dog") learns exactly the same way: by trying things, observing what happens, and gradually figuring out the best strategy through trial and error.

The key difference from supervised and unsupervised learning is that there is no table to start from — the data is generated by the agent's own experience as it interacts with the world.

In computing terms:

The dog = an algorithm (e.g. a neural network) that takes a state as input and outputs an action. It has internal parameters that get updated as learning progresses.

The treat = a number, yes. Literally a scalar, like +1 for a good outcome, -1 for a bad one, 0 for neutral. The algorithm tries to maximise the sum of these numbers over time.

Sitting down = an output value produced by the algorithm — for example the number 2 if "sit" is action #2 in a list of possible actions {stand, sit, roll over, ...}. The environment then reads that number and responds accordingly.

The room/situation = a vector of numbers describing the current state of the environment — for example [distance from trainer, whether treat is visible, previous action taken, ...].

So the whole loop in computing terms is:

the neural network receives a vector of numbers (state)
it outputs a number (action)
the environment receives that number, updates itself, and returns two things: a new vector of numbers (new state) + a scalar (reward)
repeat — and after each step, the network's parameters are nudged to make rewarded actions more likely in the future

25 of 27

Reinforcement learning

Chess board

Current configuration

Computer

Good or bad move

Reinforcement Learning — Chess example (introduction)

Chess is an ideal illustrative example for RL because it highlights the complexity of sequential decision-making under uncertainty.

Unlike spam filtering:

- A single email is classified in isolation — one prediction, immediate feedback

- A chess game requires ~40–80 moves, each dependent on all previous moves

- The outcome (win/lose/draw) is only known at the end of the game

- The "correct" move at any point depends on the opponent’s strategy, which is unknown in advance

What the agent needs to learn:

- How to evaluate any board configuration (good position vs bad position)

- Which moves lead to better board configurations over time

- Long-term strategy, not just immediate local optimisation

This reflects the fundamental challenge of RL: the agent must learn from sparse, delayed rewards. Early in training, the agent plays essentially randomly. Over millions of games, it discovers which sequences of moves tend to lead to wins and updates its policy accordingly.

Historical note: IBM’s Deep Blue (1997) was not a pure RL system — it used search algorithms and hand-crafted evaluation functions. DeepMind’s AlphaZero (2017) was a landmark RL achievement: it learned chess from scratch through self-play, with no human-designed rules, reaching superhuman performance in 24 hours.

AlphaZero is a reinforcement learning system developed by DeepMind (a Google subsidiary), published in 2017. It learns to play board games — chess, shogi, and Go — starting from nothing but the rules, with no human game data or domain-specific heuristics.

The key ideas behind it:

Self-play: it learns entirely by playing against itself, starting from random moves and improving iteratively.

Monte Carlo Tree Search (MCTS): at each move, it uses a search tree to explore possible futures, guided by a neural network that estimates both the value of a position (who is likely to win) and the probability of each possible move.

Single neural network: unlike its predecessor AlphaGo, which used separate networks for different tasks, AlphaZero uses one unified deep neural network for both move selection and position evaluation.

Generality: the same algorithm, with no changes, mastered three completely different games — which was a notable departure from systems designed for a single game.

In chess, it reached superhuman strength in about 4 hours of training on specialised hardware, and its playing style was considered remarkably creative by human grandmasters — it would sacrifice material and play long-term positional ideas that traditional engines would not generate.

It was a landmark result because it demonstrated that a general-purpose learning algorithm, given only the rules of a game, could surpass decades of human-crafted expertise and specialised engines like Stockfish.

This slide maps the abstract RL components onto the concrete chess example:

- Environment: the chess board — the physical (or simulated) space within which the game is played

- State: the current configuration of all pieces on the board — fully described at each turn

- Agent: the chess-playing programme (the "player" making decisions)

- Action: any legal chess move available from the current state

- Reward: in the simplest formulation, +1 for winning, -1 for losing, 0 for draw — received only at game end

The reward design challenge: defining what counts as a "good move" mid-game is non-trivial. In practice, intermediate rewards might be assigned based on heuristics (e.g., capturing a piece is rewarded), but too much reliance on heuristics can constrain learning. AlphaZero used only win/lose/draw as terminal rewards — and still achieved superhuman performance.

Learning process: the agent starts with random play. After millions of self-play games, it accumulates statistics on which moves from which positions tend to lead to wins. The policy is updated incrementally: moves that were part of winning games are reinforced; losing game moves are penalised.

Extending the analogy: the same framework applies to drug dosing in ICU patients (state = current patient vitals, action = drug dose, reward = patient improvement), robotic manipulation (state = joint positions, action = motor commands, reward = task completion), and protein sequence design (state = partial sequence, action = next amino acid choice, reward = predicted binding affinity).

Note:

Reinforcement learning = a general framework for learning through interaction and reward signals
Deep RL = RL where the value function or policy is represented by a neural network

Many practical robotics, operations research, and classic game AI applications still use non-neural RL methods perfectly effectively.

26 of 27

Reinforcement learning: 8 examples

Automated robots
Natural language processing
Marketing and Advertising
Recommendation systems
Gaming
Energy conservation
Traffic control
Healthcare

https://onlinedegrees.scu.edu/media/blog/9-examples-of-reinforcement-learning

Reinforcement Learning — 8 application domains

This slide broadens the scope beyond games to illustrate the real-world reach of RL across diverse domains. These examples show that the agent-environment-reward framework is extremely general.

Brief notes on each domain:

1. Automated robots: RL trains robots to perform physical tasks (manipulation, locomotion) by trial and error in simulation, then transferring to the real world.

Agent: the robot's control algorithm
State: sensor readings (joint angles, camera feed, distance to object)
Action: motor commands (move arm left, grip, release)
Reward: +1 for successfully grasping an object, -1 for dropping it
Environment: the physical world (or a simulation of it)
Policy: the learned mapping from sensor readings to motor commands

2. Natural language processing: RL is used in fine-tuning language models (including ChatGPT via RLHF — Reinforcement Learning from Human Feedback) to align model outputs with human preferences.

Agent: the language model
State: the conversation so far (the prompt + tokens generated so far)
Action: the next word (token) to generate
Reward: a score from a human evaluator (or a model trained to predict human preferences)
Environment: the human + the evaluation process
Policy: the learned tendency to produce responses humans rate highly

3. Marketing and advertising: RL optimises ad placement, content recommendation, and pricing strategies in real time, treating each user interaction as a state-action-reward loop.

Agent: the ad recommendation algorithm
State: user profile, browsing history, current page
Action: which ad to show
Reward: +1 if the user clicks, 0 if not (or purchase value)
Environment: the user and the platform
Policy: the learned rule "show this type of ad to this type of user in this context"

4. Recommendation systems: platforms like Netflix and Spotify use RL-like methods to learn which content to recommend to maximise long-term user engagement.

Agent: the recommendation algorithm
State: user's viewing/listening history, time of day, recent interactions
Action: which title or track to recommend
Reward: did the user watch/listen? For how long? Did they come back tomorrow?
Environment: the user
Policy: the learned strategy to maximise long-term engagement (not just the next click)

5. Gaming: landmark achievements include AlphaGo (Go), OpenAI Five (Dota 2), AlphaStar (StarCraft II) — demonstrating superhuman performance in complex strategic games.

Agent: the game-playing algorithm
State: the current board/game configuration
Action: the next move
Reward: +1 for winning the game, -1 for losing (given only at the end)
Environment: the game rules + the opponent
Policy: the learned strategy — which move to play in each board position

6. Energy conservation: Google’s DeepMind used RL to reduce cooling energy in data centres by ~40% by learning adaptive Heating, Ventilation, and Air Conditioning (HVAC) control policies.

Agent: the HVAC control algorithm
State: current temperature readings, server load, outside weather, time of day
Action: fan speed, cooling settings, airflow adjustments
Reward: energy saved while keeping servers within safe temperature range
Environment: the data centre
Policy: the learned schedule of cooling adjustments that minimises energy use

7. Traffic control: RL agents can optimise traffic signal timing across networks to reduce congestion, adapting dynamically to real-time traffic patterns.

Agent: the traffic signal controller
State: number of cars waiting at each junction, current signal states
Action: which lights to switch green/red and for how long
Reward: reduction in average waiting time across the network
Environment: the road network and the cars on it
Policy: the learned timing strategy that keeps traffic flowing

8. Healthcare: RL is being applied to personalised treatment planning (e.g., sepsis management, radiation therapy planning), where sequential treatment decisions must be optimised over time.

Agent: the treatment recommendation algorithm
State: patient's clinical data at a given moment (vital signs, lab results, medications given so far)
Action: next treatment decision (e.g., increase drug dose, apply radiation to area X)
Reward: patient survival, recovery speed, minimisation of side effects
Environment: the patient's body and its response to treatment
Policy: the learned treatment protocol — what to do next given the patient's current condition

Reference for further reading: https://onlinedegrees.scu.edu/media/blog/9-examples-of-reinforcement-learning

27 of 27

Supervised learning

Unsupervised learning

Reinforcement learning

> Labelled data

> Direct feedback

> Predict

outcome/future

The three different types of (traditional) ML

> No labels/targets

> No feedback

> Find hidden

structure in data

> Decision process

> Reward system

> Learn series of

actions

Summary — Three types of ML

This closing slide wraps up the conceptual overview of the three ML paradigms and sets the stage for the rest of the course.

Summary of key distinctions:

Supervised learning:

- Labelled training data required

- Direct feedback: error is measurable against known labels

- Goal: predict outcomes or class membership for new data

- Algorithms: logistic regression, decision trees, SVMs, neural networks, k-NN, random forests

Unsupervised learning:

- No labels required

- No direct feedback: quality is assessed indirectly

- Goal: discover hidden structure — clusters, components, compressed representations

- Algorithms: k-means, hierarchical clustering, PCA, t-SNE, autoencoders

Reinforcement learning:

- No labels; agent learns through interaction

- Feedback via reward signal (not a ground truth label)

- Goal: learn a policy that maximises cumulative reward over a sequence of actions

- Algorithms: Q-learning, policy gradients, actor-critic methods

Important note for this course: Reinforcement Learning is out of scope. It is a rich and technically demanding field with dedicated courses. We will focus on supervised learning, which is central to computational biology and bioinformatics.

What comes next: having established the conceptual framework, we will dive deeper into specific algorithms — decision trees, support vector machines, k-nearest neighbours, random forests, and neural networks — all of which are used within the supervised (and sometimes unsupervised) framework.

Suggested YouTube resource for RL examples: https://youtu.be/07Qum_mpEL0?si=XfcKgnwdeY4EJIkV (from 13:08)