1 of 52

Using machine learning models to�assess variation in fossilized horse�teeth for species classification

LESSON FOUR

2 of 52

Day 1

3 of 52

Introduction

Paleontologists study fossils, and it is important for them to be able to identify and classify fossils into different species.

4 of 52

Recall

Horses have an extraordinary fossil record in North America, from 55 million years ago to around 10,000 years ago.

5 of 52

Taxonomic Classification of Horses Today

Kingdom – Animalia

Phylum – Chordata

Class – Mammalia

Family – Equidae

Genus – Equus

Species – caballus

6 of 52

During the Miocene Epoch

Major climate shifts (cooling, more arid)
Changes in vegetation (expansion of C4 grasses)
Radiation of horses (~12 genera and as many as 12 species co-existing in an area)

7 of 52

Family (Equidae) Phylogeny

The branching pattern of evolution is evident especially when looking at the Miocene!

(Source: MacFadden, 2005)

8 of 52

Tooth Morphology Trends

Changes in climate, environment, diet, and survival and extinctions take place over the entire 55-million-year history of horses in North America.

Recall in the past lessons, we have observed how the teeth of horses provide evidence of these changes...

9 of 52

Tooth Morphology Examples

Low-crowned (brachydont) teeth High-crowned (hypsodont) teeth

Experiments with complexity of enamel folds on teeth (plications)

10 of 52

Metadata Details

Quality specimens that are useful in research (research-grade) will contain important details:

Locality information

geographic area, stratigraphic layer, age

Other fossils collected alongside it

used to reconstruct the ancient ecosystem

Morphological information

distinguishing characteristics used to identify it

11 of 52

The Importance of Field Data for Proper ID

Without these clues, isolated teeth or bones can be misidentified

This can cause incorrect species counts or misinterpretation of ecological roles and ancient habitat
Scientists are constantly revising our taxonomic and phylogenetic understanding of organisms with new data

12 of 52

Paleopopulations

When the deposition and preservation details from a particular fossil location are known, fossil samples of a species at that site can be considered an ancient population.

A population of Mesohippus, browsing in their forest habitat in this 1913 painting by Bruce Horsfall.

13 of 52

Coexistence and Biodiversity

There may be multiple paleopopulations at�a single site. Different species often coexist by:

Niche partitioning

browsing vs grazing
generalists vs specialists
differences in body size
differences in habitat preference

14 of 52

Sorting Activity Instructions

Look at the images provided and sort them into six (6) groups using any similarities or differences that you see!

15 of 52

References

Now, take a look at the species reference cards.

Equus simplicidens - basal modern horse

Mesohippus sp. - the middle horse

Dinohippus interpolatus - the terrible horse

Parahippus leonensis - the side horse

Nannippus aztecus - the dwarf horse

Sifrhippus sandrae - the dawn-first horse

16 of 52

Proper Sorting

Paleontologists divide these teeth into six species based on their morphology.

Sifrhippus sandrae - the dawn-first horse 55-45 million years ago

Mesohippus sp. - the middle horse 37-32 million years ago

17 of 52

Proper Sorting

Paleontologists divide these teeth into six species based on their morphology.

Parahippus leonensis - the side horse 23-16 million years ago

Nannippus aztecus - the dwarf horse 13-3 million years ago

18 of 52

Proper Sorting

Paleontologists divide these teeth into six species based on their morphology.

Dinohippus interpolatus - the terrible horse 13–5 million years ago

Equus simplicidens - basal modern horse

5 million years ago to present

19 of 52

What Did You Do?

Discuss as a class:

When sorting, did you use this same species-based classification, or a different one?

Which features did your group use to sort?

20 of 52

Tooth Position

Horses are diphyodont. They have two sets of teeth during their lifetime: deciduous (milk or baby teeth) and adult teeth
This study uses the adult tooth set
The adult dental pattern in horses = 3 (incisors), 1 (canine), 3 (premolars), 3 (molars)
All of our teeth from these lessons are upper first molars (circled in red)

Image Credit: https://vivo.colostate.edu/hbooks/pathphys/digestion/pregastric/horsepage.html

21 of 52

Tooth Characteristics (Morphology)

Key features on the chewing (occlusal) surface of the tooth can help paleontologists determine Family, Genus, and Species, especially given the context of the fossil locality and if other material (skull, bones) from the same individual are collected
The protocone is a key diagnostic feature

Mesohippus sp.

Nannippus aztecus

Dinohippus interpolatus

Equus simplicidens

No true protocone

Isolated protocone

Rounded/slightly elongate and connected protocone

Pronounced elongate ("wooden shoe")and connected protocone

22 of 52

Phenotypic Variation

However, be aware of phenotypic variation--the variability that can observed in a specific trait within a population

protocone

plications

These are all Nannippus aztecus, but they have normal variations in the number of plications, protocone shape, and more

23 of 52

Day 2

24 of 52

How Can Computers Help Scientists?

Sometimes, making identifications can be difficult. There might be very many specimens to look at, or details that are difficult to tell apart.

For centuries, computer scientists and mathematicians have thought of ways to use computers to help solve problems like these.

Let’s look at the history of computers in science, then consider how we might use computers in this case to help us with fossil identifications.

How old is the field of computer science?

For how long has artificial intelligence been a thing?

25 of 52

History of How Computers Think

1842: Ava Lovelace wrote about an idea for an “Analytical Engine,” a machine that could one day use math to make decisions. �

1950: Alan Turing wrote about computer programs that could solve problems similar to how a person would, if people first gave it good instructions to follow.

�1959: Arthur Samuel gave a computer instructions for playing checkers. The computer learned from these instructions, as well as from its wins and loses, and was able to apply decision making to new scenarios.

When computers can “learn” from what they have seen and apply this learning to new situations, we call this machine learning.

Let’s explore some key points in history. As far back as 1842, researchers have been interested in exploring machines and machine intelligence in order to get them to perform useful operations for humans. In 1842, Ava Lovelace wrote about analytical engines. She envisioned machines that could one day use math to make decisions. Nearly 100 years later, Alan Turing’s famous Turing Test was developed to evaluate a machine’s ability to exhibit intelligence-like behavior. By 1959, Arthur Samuel developed the first rule-based algorithm for humans and computer checkers play. These concepts were the start of artificial intelligence and machine learning. AI and machine learning have seen several boom periods over the past ~150 years including the huge surge happening today due to faster compute power, the internet, and more.

26 of 52

Early Computer Vision Projects

In 1966, professor Seymour Papert planned to use a camera to give visual input to a computer, which would then look at the pixels and make decisions, dividing the image into sections like “object” versus “background.”

This was one the earliest experiments with computer vision, a type of machine learning.

(He later used what he knew of computers to �program robots that could draw pictures!)

27 of 52

What is Intelligence?

So computers can learn from things they have seen, which can help us do research!

Does this mean that computers are intelligent? Which of these have intelligence?

28 of 52

What is Intelligence?

So computers can learn from things they have seen, which can help us do research!

Does this mean that computers are intelligent? Which of these have intelligence?

29 of 52

Artificial Intelligence

Intelligence describes our ability to use logic and problem solving to adapt.

When computers are able to learn from repetition to adapt, we call this artificial intelligence, or “AI.”

30 of 52

Machine Learning Models

One potential way to quickly sort fossil teeth is by training a machine learning model to recognize their complex features

Remember:

Artificial Intelligence - technology that enables computers to learn like humans

Machine Learning - Subset of AI, learns from data to make predictions

Artificial Intelligence

Machine Learning

computer vision

31 of 52

What Do You Think?

What do you know of that uses AI?
Which of those are machine learning?

Artificial Intelligence

Machine Learning

computer vision

I’d like to hear from you all now…What do you know of that uses AI? Which of these are examples of machine learning? Computer vision?

Note: As students give answers, teacher points to where in the concentric circles those ideas fall.

Broad AI examples:

Google translate, Microsoft translate (uses a type of machine learning known as deep learning)

Siri, Alexa, Google Assistant (uses machine learning through speech recognition and deep learning)

Netflix, YouTube, Spotify, Amazon (uses machine learning through text prediction)

Tesla Autopilot, driver assist systems in vehicles (uses computer vision and deep learning decision models)

Medical imaging tools, diagnostic systems, Google Teachable Machine (uses computer vision and prediction)

Online support chatbots on websites (large language machine learning models)

Fraud detection in banking (uses machine learning to detect patterns)

iPhone face recognition, social media tagging, Seek and iNaturalist (uses computer vision and deep learning)

Non-player character behavior in video games (not machine learning–rule based learning)

Business analytics and data analysis by companies (uses machine learning for prediction, but rule based learning for analytics)

Smart thermostats, door locks, lights, and security systems (uses rule based learning and some machine learning to adapt to behavior)

32 of 52

Computer Vision

Humans

Computers

Shape, Color, Features

Pixels, Textures, Predictions

Training computers to understand our visual world…

33 of 52

Pixels Explained

Notice how the image is read based on what is present in different pixels. This is different from how we might read this image.

https://visionbook.mit.edu/convolutional_neural_nets.html

34 of 52

Chihuahua or Muffin?

35 of 52

How Computer Vision Works

Pixels explained:

The atomic (smallest) unit of a digital image
Represented as squares in the image
Each square contains a value

Black and white images (values from 0 to 255)
Color images (RGB creates up to 16 million color variations by these combinations)

When humans look at an image, we often use the shape of the object, its color, and particular features to identify it. When a computer reviews images, it uses the individual pixel values and textures in the image to make predictions about it. Each pixel converts the objects in an image into numerical values where grayscale images contain values from 0 (black) to 255 (white). We just saw those numeric values for the pixels with our chihuahua and muffin on the last slide in grayscale. For color images are typically composed of the primary colors – Red, Green, and Blue (RGB) each of these colors contain 0 to 255 pixels and when the three color layers are combined- the total possible color variations contain up to 16 million combinations. This image on the right of a dog shows the conversion to pixel values for a computer vision model.

36 of 52

Convolutional Neural Networks

Convolutional neural networks are used to train a computer to identify features in images

Adapted from: https://www.ionos.com/digitalguide/websites/web-development/convolutional-neural-networks/

We will be using a computer vision based machine learning model called a simple convoluted neural network to train the computer to identify features from horse tooth images to detect patterns for classifying them by species. A convolutional neural network, or CNN, works kind of like our own nervous system which is why it is called a neural network. Again, each image is made up of tiny squares or pixel values that are input into the model, it uses layers like filters to detect textures or patterns across the pixels. Some filters look at edges, other filtered look at curves or spots and so on. It pools the most important features for each of these layers and stacks them and does this filtering again and again until ultimately making a prediction, or best guess, as to what the image is. All of this is done by transforming the image data into numerical values and running mathematical calculations.

37 of 52

Supervised Learning

Supervised learning:
We are using supervised learning for the next activity.
Scientists must first correctly identify fossils
The fossils are labeled
The computer model trains on the labeled data to find patterns to correctly identify species.

From: https://huda-kassoumeh.medium.com/supervised-learning-vs-unsupervised-learning-37e0bfb02a88

38 of 52

Unsupervised Learning

Unsupervised learning:
Data remain unlabeled or sorted
The computer model trains on unlabeled data and must cluster the data based on patterns.

From: https://huda-kassoumeh.medium.com/supervised-learning-vs-unsupervised-learning-37e0bfb02a88

39 of 52

Supervised vs. Unsupervised Learning

When we sorted the teeth the first time yesterday, was this supervised or unsupervised?

What about when we got the key?

What are benefits or limitations to unsupervised learning?

40 of 52

Training Loss and Accuracy

Used to evaluate the model’s performance

loss

(iterations)

Now I’ll introduce the three tools that we will use to evaluate the model: training loss & accuracy, confusion matrix and explainable AI. During the training the model phase, we run several epochs or iterations using a training dataset of labeled images. We can review training loss which measures the prediction error of the model, while training accuracy calculates the percentage of correctly predicted instances out of all the data in the dataset to understand how our model performs. Together they indicate how well the model is able to generalize when confronted with unseen (new) data. You want to train the model until you reach peak performance in accuracy. When the accuracy starts to drop, you have started to overfit the model, which means it now memorizing the data and is no longer able to generalize well.

41 of 52

The Confusion Matrix

Shows where a model is successfully making predictions and where the model is making mistakes.

42 of 52

Explainable AI (XAI) – Grad-CAMS

XAI: Tools and methods used to understand and trust decisions made by machine learning algorithms.

Grad-CAM: A heat-style map that highlights the areas in the input image with the greatest influence for the model’s output prediction.

43 of 52

Hear From Scientists

Some paleontologists use traditional methods in their work, while others use machine learning models and emerging technologies. Others use both! In these videos, you will hear from scientists on how they do their work.

Stephanie Killingsworth (traditional) Dr. Caleb Gordon (emerging technologies) Dr. Katie Wolcott (emerging technologies)

44 of 52

STEM Careers Using AI

Biological Sciences and Medicine
Conservation Ecology & Environmental Sciences
Astronomy
Geology & Earth Systems Sciences
Physics
Agricultural Sciences
Marine Sciences
Atmospheric Sciences

This lesson focuses on one way in which AI and machine learning can be used in the field of paleontology. The fact is, most scientific fields and STEM areas/careers today are using various forms of AI and machine learning to ask and answer scientific questions. This is a short list of broad scientific disciplines using AI today. Can you think of ways in which some of these professions might use AI?

Note: Assign the articles among students and allow them to read and then present from their groups Then: Now we have seen some examples for how machine learning works in different sciences. Scientists may use machine learning when there is a lot of data to process or when they need to work quickly. These articles also discussed some concerns that exist with machine learning, such as the risk of error and the costs of training. Tomorrow, we’re going to take a full look behind the scenes at how the coding for a machine learning model works, to better understand the ways it can be applied in paleontology.

Note: The following articles can be used for jigsaw activity in Day 2:

Machine learning in conservation and biology: https://theconversation.com/thousands-more-species-at-risk-of-extinction-than-currently-recorded-suggests-new-study-188243
Machine learning in climate studies and meteorology: https://theconversation.com/ai-powered-weather-and-climate-models-are-set-to-change-the-future-of-forecasting-235186
Machine learning in astronomy: https://theconversation.com/ai-is-helping-astronomers-make-new-discoveries-and-learn-about-the-universe-faster-than-ever-before-204351
Machine learning in museums and botany: https://theconversation.com/botanical-time-machines-ai-is-unlocking-a-treasure-trove-of-data-held-in-herbarium-collections-253236
Machine learning in environmental science: https://theconversation.com/nanoparticles-and-artificial-intelligence-can-help-researchers-detect-pollutants-in-water-soil-and-blood-271149

This concludes Day 2 content.

45 of 52

Day 3

46 of 52

Day Three

We are going to run a supervised machine learning model to explore how it identifies upper molar teeth from six horse species (representing over 55 million years of the fossil record!)

Hyracotherium, Mesohippus sp., Parahippus leonensis, Nannippus aztecus, Dinohippus interpolatus, and Equus simplicidens.

You will compare your model results to how your group classified the horse tooth images provided at the start of the lesson. If time allows, you can also try adjusting parameters and rerunning the code.

47 of 52

Let's Get Started!

Machine learning models take time to run, so let's get started.
Go to mybinder.org
In the first box, type or paste in:�https://github.com/stephkillingsworth/CoC_Chapter4.git

Type the URL here

Then click "launch"!

48 of 52

Launching Lesson 4

Click on this option to �open the notebook

49 of 52

Running Code Cells

Input Answers on Your Student Worksheet

You should now see the Jupyter Notebook. Use the play button at the top or the Shift + Enter keys to run each section of code.

You must wait for the previous cell to complete and run all cells in consecutive order.

When a cell is running or busy, you will see an * in the bracket at left of the cell. When it has completed the run it will read idle at the bottom of your screen and the bracket will show a number (example: [2] – The actual number is not important.

Some sections will take about 5 minutes to run. Record your observations in your worksheet while you work through the code.

When you have run all the code and are ready to change parameters and re-run, you must first restart the kernel (↻ beside the run keys at top) and re-run all cells from the start with your new changes added.

Now your notebook is ready. Each cell contains code and those cells must be run one after the other. Some cells will take a bit longer than others too. To run a cell you can use the shift and enter keys or the play button at the top as seen on this slide. When a cell is running, you will see an asterisk (*) at the left side of the cell and it will state “busy” at the bottom left corner of the screen. When you can proceed, a number will appear next to the cell, sometimes some output will be provided after the cell, and you will see it will state “idle” at the bottom left corner. We will be running through this code together to learn what is happening. Be sure you are keeping up with the group and have your worksheet handy to write down information from your Jupyter Notebooks when you are signaled to do so. When we see a magnifier in the notebook, this signals for us to write information onto our worksheets.

50 of 52

Tour the Image Files

Six classes (species)

images

51 of 52

Remember

Recall how you sorted teeth based on their features
Recall some of the ways scientists use both traditional and emerging methods in paleontology

52 of 52

Discussion

How well did your first model perform? What happened when you changed a parameter?

Overall, how do you think the machine learning model did at identifying the fossil teeth?

Do you think machine learning models are useful tools in paleontology? Why or why not?