1 of 75

Predicting Health Outcomes

Correlation Matrices

2 of 75

What are the indicators of healthy hearts?

Respond to questions here: https://bit.ly/CA-Pre-Assessment

Pre-class suggestions for time-savings: Ask students to think about or assign homework, the day before beginning the lesson:

Take the Pre-Assessment: https://bit.ly/CA-Pre-Assessment (they will be answering the same questions in the Post-Assessment, so answering fully and to the best of their ability, even if its a guess, will help measure how much they have learned during the lesson.)
Download and make a copy of this to use during the ML Correlation activity:�Sheets: Student Correlation Activity-Sheets; Teacher Correlation Activity-Sheets

Excel: Student Correlation Activity-Excel; Teacher Correlation Activity- Excel

Start of First Class,

Begin a discussion by asking: “What are the indicators of healthy hearts? What do you know about heart health?” After calling on several students to help compile a short list of their ideas and familiarity with the disease.
Gather student ideas – some may say comparing healthy hearts to unhealthy ones - or “statistics” or “first hand observation” – eventually focus on this: Ask: How do we know? Where does this heart health information come from? (answers may vary - statistics or risk factors)
Where do these statistics/risk factors come from?

“Yes, we can look at patient data to study the variations in a population so we learn about what may lead to heart disease.” Studies (to compare variables in populations) are done using machine learning.

So, what is machine learning?

3 of 75

How is machine learning different from statistics?

Studying a whole population to determine patterns that lead to correlations about heart health - how is this done?

Scientists are using machine learning. What do you know about this so far? (Where have we seen an example of this in our course? Loop back to previous lesson - Yes, like we discussed when we looked into the P100 wellness study) This is similar to something you may have worked with in a statistics class, or learned about before. But it is not the same - what are the differences between machine learning and statistics?

While both statistics and machine learning involve creating models based on data, machine learning models emphasize prediction while statistical models focus on estimation and inference (a conclusion based on evidence and reasoning).

Statistical concepts are commonly used in machine learning, but machine learning practitioners are free from model assumptions that statisticians have to grapple with (like what is the distribution of the population we are collecting data from).

In other words, as long as the machine learning model can make good predictions, they can use any techniques they want to develop their model, whereas the statistical models a heavily reliant on the type of data they’re concerned with, and are not necessarily used to make predictions (or at least accurate ones).

https://towardsdatascience.com/the-actual-difference-between-statistics-and-machine-learning-64b49f07ea3

https://www.svds.com/machine-learning-vs-statistics/

These are two helpful articles to help students with the above question.

4 of 75

How are machine learning and artificial intelligence similar to each other? How are they different?

Machine learning and artificial intelligence often appear in the same context, so it is easy to confuse the two as the same thing. However, while their applications definitely overlap, there are key differences that we must note.

Ask: What are the similarities and differences of machine learning and artificial intelligence? Some students might not come up with answers to these questions, and that is completely fine.

Machine learning is considered to be a subset of artificial intelligence. Artificial intelligence is the ability of a computer/other machine to mimic human activity and thinking (i.e. a chatbot that mimics human conversation and emotions). Computers use machine learning algorithms, mathematical concepts, and logic to help computers complete tasks and problem solve like humans would. Machine learning is the means by which AI gets its intelligence.

Although artificial intelligence is one crucial application of machine learning, let’s think of examples of machine learning as it is utilized, and in many instances outside of artificial intelligence.

The activity we are about to do would not be classified as artificial intelligence because it is not mimicking human activity.

5 of 75

What does Machine Learning do?

6 of 75

In what everyday scenarios can machine learning be used?

7 of 75

Streaming Platforms

8 of 75

Marketing and Advertising

9 of 75

Transportation

10 of 75

Banking and Finance

11 of 75

Historical Research

12 of 75

Music

13 of 75

Natural Language Processing

14 of 75

Biology Research

Machine learning can be used to classify cells as dead or alive, mining genomic data (for example, identifying single-nucleotide polymorphisms), and flagging RNA as mutated or pathogenic. Drug and vaccine discovery is another area of focus: machine learning can determine the 3D structures of proteins and other molecules, which helps researchers create drugs that better target proteins of interest.

As another example, a predictive machine learning algorithm is being used to more accurately forecast which flu strains will be predominant in the future. This can help scientists develop vaccines for future flu strains and reduce mortality rates.

Source: https://www.nature.com/articles/d41586-018-02174-z.�https://wp.technologyreview.com/wp-content/uploads/2022/02/tr10-ai-protein-folding-16x9-1.png?w=2256

15 of 75

Medicine

In medical settings, machine learning is used to enhance personalized, precision medicine: predicting what treatment protocols are likely to succeed on a patient based on various patient attributes and the treatment context; as well as diagnosing a patient.

Example: Sevda Molina, PhD - who helped develop these lessons - uses machine learning to research populations affected by inflammatory disease such as multiple sclerosis. She is looking at not only genetics, but also longitudinal patterns in both phenotype (clinical observations and patient reported outcomes) and exposures (such as anti-immune response medications, infectious disease and health-related social needs).

Machine learning (specifically, neural networks/deep learning) is also used to classify tumors as malignant or benign and/or classify images in oncology or even forecasting flu strain prevalence, during pandemics.

Additionally, natural language processing algorithms can be used to transcribe conversations between doctors and patients or analyze clinical notes.

Source: https://miro.medium.com/max/978/1*WmGCezZGdyhX4ahjAm1p4g.png

https://healthitanalytics.com/news/predictive-analytics-models-forecast-prevalence-of-flu-strains

16 of 75

“Cardiovascular disease - like heart attacks and stroke - may seem like an elderly disease. But studies have shown that cardiovascular events are happening more and more frequently in people as young as 20.”

One example we will use for the activities in this module is predicting risk for cardiovascular disease. “Cardiovascular disease - like heart attacks and stroke - may seem like an elderly disease. But studies have shown that cardiovascular events are happening more and more frequently in people as young as 20.” To assist in preventing this - Machine Learning is used to see patterns in large datasets from a population that would otherwise not be possible.

A Synopsis of a recent heart disease study that will be the focus of our activities:

Even though fewer heart attacks are occurring in the U.S., these events are steadily rising in very young adults. New data not only validate this trend but also reveal that more heart attacks are striking those under age 40, according to a study presented at the American College of Cardiology’s Annual Scientific Session (2019).

“It used to be incredibly rare to see anyone under age 40 come in with a heart attack—and [...] now [they are] in their 20s and early 30s,” said Ron Blankstein, MD, a preventive cardiologist at Brigham and Women’s Hospital, associate professor at Harvard Medical School in Boston and the study’s senior author. [....]

The study is the first to compare young (41-50 years old) to very young (40 or younger) heart attack survivors.

Among subjects who suffer a heart attack at a young age overall, 1 in 5 is 40 or younger.
During the 16-year study (2000-2016) the proportion of very young people having a heart attack rose 2 percent each year.

Reflect: Have students Pair-Share and then discuss as a class briefly.

What do you notice or wonder about?
What do you think might be some risk factors for cardiovascular disease?
Why do you think the disease might be trending younger?

We will be using machine learning, with a large dataset from the Framingham heart health study – to find factors to help answer the questions:

What are the indicators of healthy hearts?
What steps can researchers and doctors take to monitor and improve the health status of our cardiovascular system?

Possible Extension: Have students read information about the increase in cardiovascular events in younger people. Or explain briefly the background of the study using synopsis here.

NOTE: Do not get into too much detail at this point - there will be time at the end of the module to compare student predictions and current CDC information – after they have made their own predictions using machine learning – students are invited to examine CDC interactive map of heart disease.

Source: https://www.acc.org/about-acc/press-releases/2019/03/07/08/45/heart-attacks-increasingly-common-in-young-adults

17 of 75

P100 wellness study correlations

Framingham dataset correlations

Observe two circos plots of correlations — on left is a familiar one from the P100 wellness study, the other is a small subset of data from the Framingham heart health study. Compare the two images - what can we learn from these? In the P100 wellness study (left), (Module 2: Lesson 2), we saw that researchers often find correlations (between microbes and disease, for example) and use that information to help people better their health, lower their risks, and improve their health outcomes. We’re going to do something similar with heart health.

Through this activity, we’ll determine what variables could have an impact on or be a risk for heart disease and how to predict the risk of a patient getting heart disease. The figure on the right is just a small subset of the framingham data -

Why would the correlations on right appear different than the plot from the P100 wellness study? [ A: fewer dataset in the study]

We’ll use a publicly available dataset - the Framingham heart study dataset - to create a model that determines a patient’s risk of getting heart disease.

First, we’ll create a correlation matrix, just like in the P100 wellness study, to determine which variables have the most correlation with cardiovascular disease (CVD).
Then, we’ll use the correlation information to create a logistic regression machine learning model that determines a patient’s risk for heart disease - just like a research scientist might do!

(NOTE: The Framingham correlation matrix circos plot shown here is not accurate - it was generated using only the “Small group dataset” as an example - students will create their own circos plot using a “Large Framingham dataset” at the end of this lesson sequence)

18 of 75

Correlation or Causation?

Liam collected data on the sales of ice cream cones and air conditioners in his hometown. He found that when ice cream sales were low, air conditioner sales tended to be low and that when ice cream sales were high, air conditioner sales tended to be high.

From this scenario what can you conclude? Is the relationship between ice cream cones and air conditioners a correlation or causation? Call on and check with students. What do they think?

Liam can conclude that sales of ice cream cones and air conditioner are positively correlated.

Liam can't conclude that selling more ice cream cones causes more air conditioners to be sold.

It is likely that the increases in the sales of both ice cream cones and air conditioners are caused by a third factor, an increase in temperature!

The P100 wellness study and other Machine Learning starts with finding correlations between variables.

Source: https://www.khanacademy.org/test-prep/praxis-math/praxis-math-lessons/gtp--praxis-math--lessons--statistics-and-probability/a/gtp--praxis-math--article--correlation-and-causation--lesson

19 of 75

Correlation or Causation?

Correlation: there is a relationship or pattern between the values of two variables.

Example: Liam found that when ice cream sales were low, air conditioner sales tended to be low and that when ice cream sales were high, air conditioner sales tended to be high.�

Causation: one event causes another event to occur. Causation can only be determined from an appropriately designed experiment. In such experiments, similar groups receive different treatments, and the outcomes of each group are studied. We can only conclude that a treatment causes an effect if the groups have noticeably different outcomes

Example: Liam did another study and found that when ice cream sales were low, temperature was low and that when temperatures were high, ice cream sales were high, and air conditioner sales tended to be high.

20 of 75

3 types of correlation:

Positive correlation

As x increases, y tends to increase.

Negative correlation

As x increases, y tends to decrease.

No correlation

As x increases, y tends to stay the same or have no clear pattern

Ask for volunteers to name each type as you click through this slide showing each kind.

There are three main types of correlations: positive, negative, and neutral.

It’s important to note that we can also have nonlinear correlations - we’ll discuss this concept in greater depth later in the activity.

Think, pair, share - in small groups come up with real-world examples for each type of correlation. Have students come up with a list of biological correlations - correlations they’ve seen in the P100 wellness study, correlations they know about in their everyday lives, or correlations that they’ve researched during class.

If additional conversation about correlation vs. causation is needed, have students look through this website: https://www.tylervigen.com/spurious-correlations.

Source: https://www.khanacademy.org/test-prep/praxis-math/praxis-math-lessons/gtp--praxis-math--lessons--statistics-and-probability/a/gtp--praxis-math--article--correlation-and-causation--lesson

21 of 75

By the end of this activity, you’ll create something like this!

“What are the indicators of healthy hearts?” In today’s activity we are going to explore how correlations are calculated and how to interpret the outcomes of those calculations. Our goal is to identify correlations across variables using the Framingham dataset - a human health study done over many years (a longitudinal study). To see if there are real world correlations that are repeated in populations. We will determine whether there is a correlation between different features and between features and CVD outcome.

Ask, “Why would we want to do this?” (Use their responses to this question to formatively assess the learning so far.) These correlations will help us in formulating a hypothesis about what variables have the greatest influence on cardiovascular disease and selecting features that will be used to predict health outcomes.

We will be doing these activities in Excel or Google Sheets using a calculator written for you, embedded in the sheet. But in the real world, scientists would do the calculations using languages like Python and R. These languages are designed for statistical analyses and have add-ons (packages) that allow you to quickly create correlation matrices and run logistic regression on your data.

22 of 75

Feature vs. Variable

Definition:

23 of 75

The Framingham Heart Study Background

We will be working with the Framingham dataset to look at cardiovascular disease, or CVD. What is this dataset?

The dataset you will be working with came from a long epidemiological study of a population from the town of Framingham, MA. The heart study began in the 1948, and continues to this day.
The town was picked as a study site because it was big enough to provide a sufficient number of individuals for the study, while also being small enough to use a community approach for recruiting participants.
Participants came from a range of ages, both male and female.
Blood samples and health history were taken and the participants heart health was followed over many, many years.
The study has expanded geographically and in population scope and continues up until today.
You will use a subset of the Framingham study, which includes laboratory, clinic, questionnaire, and health event data on 4,434 participants. Data was collected during three examination periods, about 6 years apart, from 1956 to 1968. Each participant was followed for a total of 24 years to see if there were any of these CVD outcomes: Angina Pectoris, Myocardial Infarction, Atherothrombotic Infarction or Cerebral Hemorrhage (Stroke) or death.

While doing these activities, keep in mind that the results achieved from these activities are within the boundaries of this Framingham dataset.

24 of 75

How can machine learning be used to accurately predict a patient’s risk of developing CVD?

Set the stage. Ask students to “imagine we are in the 1950s…we don’t know the answers, and heart disease is one of the leading causes of death.”

Ask: What do we need to do to uncover how age is a factor? (Students may offer that we first need to ask: What factors contribute to the risk for developing Cardiovascular Disease (CVD)? So, to do this you will identify correlations, then, learn to use Logistic Regression to test predictions about the risk factors for Cardiovascular Disease. At the end we will look at these questions:

a) How do our own findings compare to the Framingham Heart Study findings?

b) What did we learn about why young people are at increasing risk for heart disease?

If not done already, assign Student Practice Activity-Sheets /Teacher Key-Sheets (Student Practice Activity-Excel / Teacher Key-Excel) - This is a good basic review of Excel / Sheets functions and skills needed for all of the Machine Learning activities to come. Use this to evaluate student’s skills to form teams to provide peer-to-peer support during all of the activities.

25 of 75

Address “null variables” and “outliers”

Classify variables

Select features

Visualize your data

Calculate correlation

Correlation Roadmap

1

2

3

4

5

Now that we have a little background on our data let’s start by using a small subset of the study data to understand correlations. Here is a roadmap of the steps a scientist would take to determine whether there is a correlation between two features (aka variables). We will follow this same process, introducing the steps for the activities as we move along the roadmap. The first two steps in orange are known as pre-processing steps.�Open the Features Key

First let’s learn about different types of variables - Ask student table groups:

Look at our Framingham dataset, what are the variables? (eg .What does SYSBP stand for [systolic blood pressure]?
What were some of the variables in the P100 wellness study? Are they similar or different?
The correlations identified in the dataset will help to predict which features have the greatest influence on cardiovascular disease as a health outcome. We will determine whether there is a correlation between feature to features and feature to outcome (which is a higher health risk for Cardiovascular Disease (CVD)).

As a class, make a guess as to which features will be the most correlated to the outcome of heart disease (CVD)? Ask students in table teams to:

Hypothesize whether the variables are correlated or not.
Rank the next top 3 features that seem most correlated with CVD. (Top 3 variables that intuitively do and some that intuitively don’t have correlations to CVD?)
Share out hypotheses and rationales as a whole class.
Discuss: “How can better correlations be calculated?”

Understanding will grow as they go through the process.

26 of 75

Classify variables

1

There are two different types of variables used in the dataset: one is a continuous variable and the other is a discrete variable.

Discrete variables are variables whose values are distinct/separate values. For example, the number of students present or the number of heads or tails flipped by a coin are discrete variables. You can also think of discrete variables as variables whose value is obtained by counting.
Continuous variables are variables whose value can be any value in an interval. For example, height or weight are continuous variables. You can also think of continuous variables as variables whose value is obtained by measuring.

NOTE: For simplicity in this introductory lesson, these two types of variables are used with this dataset. For other datasets categorical variables are also possible. [eg. sex is a categorical variable in the study - and since the program does not recognize anything but numbers, the male and female variable are represented as 0 or 1. ] Another example is genetic heterozygous, homozygous dominant, homozygous recessive data - this would be categorized into 3 types, and represented by 3 values for the variable.]

For more examples of the variable types, watch this video: https://www.youtube.com/watch?v=dOr0NKyD31Q.

Source: https://static-assets.codecademy.com/Courses/Hypothesis-Testing/Intro_to_variable_types_4.png�http://www.henry.k12.ga.us/ugh/apstat/chapternotes/7supplement.html#:~:text=A%20discrete%20variable%20is%20a%20variable%20whose%20value%20is%20obtained%20by%20counting.&text=A%20continuous%20variable%20is%20a%20variable%20whose%20value%20is%20obtained%20by%20measuring.&text=A%20random%20variable%20is%20a,outcome%20of%20a%20random%20phenomenon.&text=A%20discrete%20random%20variable%20X%20has%20a%20countable%20number%20of%20possible%20values.�

27 of 75

Classify variables

1

Your turn:

Classify the variables in your

Framingham dataset

Systems Medicine, 2019

28 of 75

Address “null variables” and “outliers”

Classify variables

Select features

Visualize your data

Calculate correlation

Correlation Roadmap

✔

2

3

4

5

29 of 75

Address “null variables”

2

30 of 75

Address “null variables”

2

31 of 75

Address “null variables”

Your turn:

Find and address null values in your dataset.

2

Systems Medicine, 2019

32 of 75

Address “outliers”

2

33 of 75

Address “outliers”

2

34 of 75

Address “outliers”

2

So how do you find an outlier? We typically use a graph called a box and whisker plot. You may have seen this in your math class, but we’ll quickly review it here. A box and whiskers plot is a way of visually showing us our quantitative data and its spread. The box shows the 25th, 50th, and 75th percentiles of the data, while the whiskers show the spread of the data.

The distance from the 25th to 75th percentile is called the interquartile range (IQR). An outlier is any value above Q3 + (1.5 * IQR) or below Q1 - (1.5 * IQR).

Teacher Note: If there is a question about the 1.5 or 150% was chosen arbitrarily to best fit with our parameters of our project design. Each project design will decide its own IQR percentage, either 100% of IQR, 200% of IQR, etc, etc, based on that projects decision.

Source: https://publiclab.org/public/system/images/photos/000/032/980/original/Screen_Shot_2019-06-18_at_10.27.45_AM.png

35 of 75

Address “outliers”

2

Continuous variables only!

Input this formula into D:19

=OR(MAX(D2:D11)>D17, MIN(D2:D11)<D15)

To calculate the 25th and 75th quartiles: Use the rows at the bottom of your sheet to input the formula =QUARTILE(start of column: end of column, 1) and =QUARTILE(start of column, end of column, 3). This will calculate the 25th percentile and 75th percentile, respectively.

Then calculate the IQR using (=Q3-Q1). Using these values you can then apply the formula =Q3+(1.5*IQR) to find the upper bound (75th percentile) and the formula =Q1-(1.5*IQR) to find the lower bound (25th percentile) which you will use to define any outliers.

Remind students they need to first find the Max boundary (Q3) and Min Boundary (Q1), then find the IQR first, before they can calculate the upper and lower boundaries. Demonstrate to them how to do this in the spreadsheet.

TEACHER NOTE: if there is confusion about when to use the box and whisker “boundary” to find outliers - remind students that it is not used for discrete variables [all values are either 0 or 1 (eg. SEX)] but is used for continuous variables.

36 of 75

Address “outliers”

Your turn:

Find and address outliers in your dataset.

2

Systems Medicine, 2019

37 of 75

Address “null variables” and “outliers”

Classify variables

Select features

Visualize your data

Calculate correlation

Correlation Roadmap

✔

3

4

5

38 of 75

Calculate correlation

Positive correlation

As x increases, y tends to increase.

Negative correlation

As x increases, y tends to decrease.

No correlation

As x increases, y tends to stay the same or have no clear pattern

Pearson’s correlation coefficient closer to 1

Pearson’s correlation coefficient closer to -1

Pearson’s correlation coefficient closer to 0

3

39 of 75

Calculate correlation

3

Before we do the calculations ourselves, let’s quickly talk about how to read a correlation matrix. When reading correlation matrices, each row and column corresponds to a variable. The r-value is a number between -1 and 1. It tells us whether two columns are positively correlated, not correlated, or negatively correlated. The closer to 1, the stronger the positive correlation. For example, correlations between the same variable, like sex and sex, will always be 1, as every variable will be perfectly correlated with itself. The closer to -1, the stronger the negative correlation (i.e., the more “opposite” the columns are). The closer to 0, the weaker the correlation.

Remember we are first seeing what things are correlated in order to suggest things like cause and effect.

Are there correlations in the small dataset that immediately make sense to you?

40 of 75

Pearson’s Correlation formula

Calculate correlation

3

Pearson’s finds the coefficient “r” for each point - a value between 1 and -1. This determines the relationship of each point to the mean point of the data set. In practice the greater the codevience the great the correlation coefficient.

Answers the question: are the points in a close or a distant relationship? How scattered or distant (deviated) are these points �of data?

Sum of squared deviations (x)

Sum of squared deviations (y)

Codeviance

How do we calculate the Pearson’s correlation coefficient? We’ll use something called Pearson’s correlation formula. You do NOT need to memorize the formula or ever use it during this activity—the computer will do all this for you. We just want to give you an understanding of what the formula is calculating and what its purpose is.

Before we talk about the meaning behind the formula, let’s discuss a few key symbols. First, the x-bar and y-bar represent the average values of x and y, respectively. We would take the average of all of our x values and all of our y values within our dataset. Second, the sigma symbol represents a sum. So in this case, we would be adding up (x - x-bar) and (y - y-bar) for every x and y point in our dataset. And finally, when we square a value and then take its square root, we are taking the absolute value of that number.

The correlation coefficient formula finds out the relation between the variables. It returns the values between -1 and 1.

Source: (Teacher Resource: https://towardsdatascience.com/correlation-explained-visually-4875448e8e56)

41 of 75

Calculate correlation

What is the purpose of the Pearson’s correlation formula? It tells us how closely our data aligns with the average x and y values, or how well x and y are correlated.

How does it mathematically achieve that purpose? We add all the differences from the average values of x and y in a standardized way.

3

42 of 75

43 of 75

44 of 75

Pearson Correlation

3

45 of 75

r = +1.0

r = +0.7

r = 0

r = -0.46

As the r-coefficient changes notice the relationship of the dataset to the line, and the slope of the line. Take an educated guess as the the r coefficient value for each of these. [Click through the animation in this slide – to reveal the r values as you describe/name each example 1-4.]

Large Positive ? At r = 1 – describe the relationship of the dataset to the line.
Medium Postive? At r = 0.7 – describe the relationship of the dataset to the line.
Small Negative Correlation? At r = -0.46- describe the relationship of the dataset to the line.
Weak or no Correlation? When r = 0 – describe the relationship of the dataset to the line.

In general the coefficient is lower (closer to zero) if the correlation is lower. And the highest correlation coefficient is 1 - which is something correlated with itself.

The r-value is a number between -1 and 1. It tells us whether two columns are positively correlated, not correlated, or negatively correlated. The closer to 1, the stronger the positive correlation. The closer to -1, the stronger the negative correlation. The closer to 0, the weaker the correlation.

46 of 75

Calculate correlation

Your turn:

Create two scatter plots with your dataset. What do you notice?

3

Systems Medicine, 2019

47 of 75

Calculate correlation

3

In the first scatter plot we can see that SYSBP is a continuous variable but SEX is discrete so points are going to appear anywhere along the SYSBP however SEX is either 0 for female or 1 for male. Ask students - What is the correlation? (The correlation is pretty low, you are going to get a pretty low positive correlation - with female patients. Right now we are not asking students to calculate the statistic - even if you measure how much there is - it is going to be a low number.) Students were asked to plot the relationship between a 1) discrete and continuous variable; and 2) two continuous variables. The discrete-continuous relationships are not going to be calculated well by a linear model. When creating a correlation matrix, Pearson’s correlation coefficient will work best for the continuous/continuous relationships. In the real world, you would have to separately analyze discrete/discrete and discrete/continuous relationships using other formulas.

We’re not going to go super in-depth with the math, but just know that for discrete/discrete pairs, we use a formula called chi (pronounced “kai”)-squared. It shows you the difference between your current data and what your data would look like without any correlation. For discrete/continuous pairs, we use a formula called point-biserial correlation. This is pretty similar to a Pearson’s correlation—just adapted for a different situation.

In the second scatter plot we can see that SYSBP is a continuous variable and so is DIABP (diastolic blood pressure). What is the correlation? Ask students - Does it make sense that these two blood pressure measurements would have a correlation? As one increases due to pulse pressure on the arteries, as the heart pumps, the other would also increase. There is a linear relationship from pulse pressure – so likewise both will decrease. With some exceptions due to arterial hardening - the correlation between these features is positive.

For this activity, we’ll just use Pearson’s correlation coefficient for everything - there will be a small amount of error because of this, but nothing that will significantly impact our results.��How can we look at more than two features at a time? Remind students that searching for correlations using scatter plots works well between two features – But, finding correlations among many features gets challenging to do in a scatter plot — This is where using Machine Learning assists in looking at features altogether and comparing.

48 of 75

Calculate correlation

Your turn:

Calculate correlation using Pearson’s Correlation Coefficient with your dataset.

3

Systems Medicine, 2019

49 of 75

Address “null variables” and “outliers”

Classify variables

Select features

Visualize your data

Calculate correlation

Correlation Roadmap

✔

4

5

50 of 75

Visualize your data

4

There’s a lot of information in our correlation matrices, and it’s hard to read it all. Scientists, especially when they have large correlation matrices with lots of data, will apply data visualization tools to make sense of what they’re seeing.

Data can be visualized in different ways - in the P100 wellness study, we saw correlations expressed as thick or thin lines between variables. Scientists might also use heat maps to show positive, negative, or neutral correlations between different variables.

Source: https://46gyn61z4i0t1u1pnq2bbk2e-wpengine.netdna-ssl.com/wp-content/uploads/2018/09/corrplot.png

https://miro.medium.com/max/1034/1*2nAoUrgs-R2HAKKtSqp1ww.png

https://www.researchgate.net/publication/329260822/figure/fig1/AS:698150423650305@1543463440314/Correlation-matrix-visualizing-significant-correlations-p-001-between-microbial.jpg

https://www.tutorialspoint.com/machine_learning_with_python/images/correlation_matrix_plot.jpg

https://miro.medium.com/max/1400/1*jKlY7iYuI0XoHSx0M9UCMQ.png

https://pbs.twimg.com/media/DPomEDcX4AApj0x.jpg

51 of 75

Visualize your data

Your turn:

Create a heat map of your data correlation matrices.

4

Systems Medicine, 2019

52 of 75

Visualize your data

4

You’ll see that, when creating your heat maps, there are several groups of correlation values ranging from insignificant to significant. The ranges of these groups are based on general scientific knowledge. In the real world, you would also have to do additional calculations to determine the probabilistic significance of your correlations - but for simplicity, we’ll just use these ranges for now.

The darkest red represents high correlation (-0.7 and above, closer to -1.0). The formatting will automatically make all values between -0.3 to -0.7 pale red which is a significant correlation. All values between -0.01 to -0.3 are lighter shades and represent an insignificant correlation.

The white represents no correlation (-0.01 to 0.01)

The darkest green represents a high correlation (0.7 and above, closer to 1.0). The formatting will automatically make all values between 0.3 to 0.7 will be a grey-green. which represent slightly significant correlations. All values between 0.01 to 0.3 are lighter shades and represent insignificant correlations.

�

53 of 75

Bias, Errors, and/or Pitfalls

54 of 75

We’ve found a correlation that fits the above data… but will it hold up for a larger dataset like the one on the right?

This might lead to sampling error: a correlation that we’ve found in the small sample sets we’ve been working with might not apply across the entire dataset. For example, if in our sample data, everyone in the first 10 rows were 65+, but there were a mix of ages in the rest of the dataset (the part that we aren’t looking at), we might not accurately calculate correlations between age and other variables for the broader population.

Unless you’re studying the entire population, there’s always going to be some bias: but when you increase the size of your sample dataset, you can minimize that bias significantly. That’s why we’re going to apply what we’ve learned to an even larger version of the Framingham dataset.

After completing the Small Dataset activities, lead the whole class in a short discussion.

Remind them the dataset used in this first activity is very small, so the correlations identified may challenge what is known about other health correlations.
Ask students and discuss: “How can better correlations be calculated?” Acknowledge that using larger datasets will be the next step.

55 of 75

Let’s apply what we’ve learned to an even larger dataset!

56 of 75

Your turn:

Create a correlation matrix using a larger dataset. �Then visualize the data.

Systems Medicine, 2019

Have students create a correlation matrix using this data Student Correlation Activity-Sheets/Student Correlation Activity-Excel Go to “2a_Large Dataset” tab first - to see the data before it has been processed. Then click to “2b_Calculation & Visualization” tab, to insert and copy the Pearson’s Correlation Coefficient to all features. Then, as before, use the 3-color Conditional Formatting to visualize the dataset.

Check their work using the “2b_Calculation & Visualization” tab on Teacher Correlation Activity-Sheets/Teacher Correlation Activity- Excel. Students can do this individually or in pairs but should check their work with their small groups at the end.

Ask students:

What variables were the most and least correlated? Does this make sense?
Were there any correlations that surprised you? Why or why not?

�

57 of 75

Calculate correlation

3

Our outcome of CVD

58 of 75

So what?

59 of 75

Address “null variables” and “outliers”

Classify variables

Select features

Visualize your data

Calculate correlation

Correlation Roadmap

✔

5

60 of 75

Select features

5

61 of 75

Select features

5

But there’s another step to feature selection. There are two types of correlations in our matrix: correlations between two features—like between glucose levels and diabetes—and correlations between a feature and our outcome of cardiovascular disease (CVD). If we want to predict our outcome, would we want to use features that are closely or not closely correlated with the outcome? Have students share their thoughts and explain their reasoning.

Because correlations show relationships between two variables, we would want to maximize the correlations between a feature and our outcome. But what about correlations between features? When we have a very high correlation between features, that actually skews our analysis.

Source: https://www.kdnuggets.com/2019/10/feature-selection-beyond-feature-importance.html

62 of 75

Select features

Orange sections: A features contribution in explaining the outcome

Red section: Mutual information provided by both features.

5

Purpose: Choose the features with higher correlation with the outcome to be able to explain it more. The goal of the whole model is to help to explain the outcome correctly, so we need features that have high correlation with the outcome but low correlation among themselves.

1. Feature to outcome correlation: We want to choose the feature in a way that it maximizes the orange sections in the Venn diagram (left). So we choose the features that have high correlation with the outcome (these features can explain or predict the outcome better). (Absolute value for feature-outcome correlation should be more than 0.05)

2. Feature to feature correlation: We want to minimize the red section to have the minimum redundant or repetitive information. So we choose the features that have a small red sections with other features (Low correlation between the features).

Note that with high correlation among the features as seen between Feature 1 and 2 (above right) (or high mutual information) we would end up giving higher weight overall in the model to those features. This could be misleading for the algorithm, if one feature is less predictive of the outcome. (Absolute value for feature-feature correlation should be less than 0.7)

3. Final Feature selection: in a case of high feature-feature correlation, we need to remove one of the features. The feature with lower correlation with the outcome will be removed, because it will contribute less to explaining the outcome. In the Venn diagram (right) Feature 1 has higher correlation with the outcome (greater Orange overlap) than Feature 2.

63 of 75

Select features

Weight

Age

BMI

Model

Weight

Age

5

64 of 75

Select features

5

65 of 75

Select features

5

66 of 75

Select features

5

67 of 75

Select features

5

68 of 75

Select features

Your turn:

Write down features correlated with each other (feature to feature) and features that are not correlated with CVD (feature to outcome).

5

Systems Medicine, 2019

69 of 75

Select features

5

Using the list of Irrelavent and redundant features- Go to the “2c Feature Selection” tab.

Here you can delete the values from the features on your list, by deleting the values from both its column and row.

Replace Null cells with a “ -”

70 of 75

Select features

Your turn:

Remove variables correlated with one another. And create a Circos plot.

5

Systems Medicine, 2019

Have students remove any values for the features correlated with one another using the “2c_Feature Selection” tab - in the Student Correlation Activity-Sheets/Student Correlation Activity-Excel

Purpose of Feature selection:

We want to remove features irrelevant to the outcome CVD. (correlation >0.01)
We want to remove redundant features - that are not adding any new information– choose one feature of two that are both highly correlated feature to feature, which indicates that they might be redundant.
By deleting the other feature, the extra weight of two features (both very highly correlated) is reduced.

Create a circos plot of the data using the Circos tab. Circos Webpage See tabs on the Teacher Correlation Activity-Sheets/Teacher Correlation Activity- Excel for steps and what this should look like when completed.

Creating a circos plot allows us to better visualize the correlations between the different features before we move into the logistic regression part of the activity.

Ask: What do students notice about their data in this circos plot format?

�

71 of 75

Circos Plot Visualization

72 of 75

What are the indicators of healthy hearts?

Let’s return to our initial question: “What are the indicators of healthy hearts?” Has this activity changed your perception of heart health? Why or why not?

If you were to create a health plan to maximize cardio-vascular disease(CVD) health, what features, variables would you want to consider?* [This is finding correlations]

Now that we know possibilities for cardiovascular disease, we might be asking “What will happen for my type of person…?”

Is correlation causation? We do not know causation yet. Why not? [Because correlation can lead us toward causation, but can not be proven without more evidence], what we have done is prepare data so that it ready for the prediction phase. We can predict that some variables are more likely to connect to cardiovascular disease as an outcome.

The dataset used in this activity is very small, so the correlations identified may challenge what is known about other health correlations. However, the purpose of this activity if for students to understand the process and begin to analyze the results related to CVD. This is a great opportunity to discuss how better correlations can be calculated using larger datasets.

73 of 75

Address “null variables” and “outliers”

Classify variables

Select features

Visualize your data

Calculate correlation

Correlation Roadmap

✔

74 of 75

Logistic Regression!

Next step…

Next we will use logistic regression to create a mathematical model to classify patients as likely or unlikely to develop CVD.

We’ll be using the features we’ve selected to analyze the data further, by creating a model using logistic regression to test a larger dataset, that will predict a patient’s likelihood of getting CVD. [our outcome]

Remind them the correlations and predictions they find among the features could then be used to [in the next step]:

Conduct research to identify causation. A Randomized control trial is the best way but, trials are expensive and take a long time to identify direct relationships; so the first step is finding and prioritizing which correlations to test - in other words making predictions!
Use correlations to analyze clinical results to predict outcomes for a new patient to inform their potential changes to lifestyle and behavior. The lifestyle and behavior changes are explored in the personal health behavior planning Lesson, later in this Systems Medicine module.

75 of 75

What are the indicators of healthy hearts?

Respond to questions here: https://bit.ly/CAPostAssessment