Part 01

Introduction to Hypothesis Testing

Fardina F Alam

CMSC 320: Introduction to Data Science

Topics We Will Cover

CMSC320 Textbook Chapter: Chapter 8

“Data alone is not interesting.

It is the interpretation of the data that we are really interested in.”

Hypothesis Testing (Making Informed Decisions with Data)

Hypothesis Testing (HT)

Hypothesis testing is a statistical method used to make informed decisions or draw conclusions about a population based on a sample of data.

HOW HYPOTHESIS TESTING WORKS

1. Make assumption about population

2. Use sample to test assumption

Population: The entire group that you are interested in studying.

Sample: A subset of the population used for analysis.

Main Steps of Hypothesis Testing

1. State the Null Hypothesis
2. State the Alternative Hypothesis
3. Pick a Level of Significance 𝛂
4. Choose a Test
5. Collect Data
6. Calculate a test statistic
7. Calculate P-Value and compare with 𝛂
8. Draw a Conclusion

Collect Data

Obtain a representative sample from the population.

Remember the importance of recognizing whether data is collected through an experimental design or observational study.

Sampling Method

Random sampling� • Every unit has equal chance of selection.

A sampling method is a process by which individual items/event (observational units) are selected from the population to be included in the sample. Common sampling methods:

Stratified sampling� • Population divided into meaningful groups (strata).� • Random samples taken from each group.

Cluster sampling� • Population divided into clusters (e.g., locations).� • Randomly select clusters and include all units inside.

Systematic sampling� • Random start → select every kth unit.

Convenience sampling (non-probability)� • Select easiest or most available participants.� • Fast but may introduce bias.

Examples: Sampling Methods

​

• Random sampling: Choosing names randomly from a list for a survey
• Stratified sampling: Surveying transportation preferences involves dividing the population into three income strata—low, middle, and high-income households—and then randomly selecting households from each stratum in a city.
• Cluster sampling: Estimating the average income of households in a city by dividing the city into clusters based on neighborhoods or districts, then randomly selecting specific neighborhoods as clusters and surveying all households within the selected neighborhoods.
• Systematic sampling: Selecting every 10th person from a list of customers
• Convenience sampling: Surveying people in a shopping mall (ease of access, availability), Recruiting volunteers from a specific organization or community etc.

Two Hypotheses in Hypothesis Testing

Hypothesis: a claim we test using data (about a population).

• Null hypothesis (H₀)

no effect / no difference (current assumption)

• Alternative hypothesis (Hₐ)

real effect / difference (opposite of H₀)

​

Possible Outcomes of a Test

1. Reject H₀ Evidence suggests a real effect or difference (support Hₐ)
2. Fail to reject H₀ Not enough evidence to support Hₐ

(We never “prove” H₀ true.)

Example: Candy Machine H₀ & Hₐ are mutually exclusive

Let’s say, it is believed that a candy making machine makes chocolate bars that are on average 5 gram in weight. A worker claims that the machine after maintenance no longer makes 5 gram bar. Write down H₀ and Hₐ.

​

Practice: Doctors believe that the average teen sleeps on average of no longer than 10 hours per day. A researcher believes that teens on average sleep longer. Write down H₀ and Hₐ.

Significance Level (α): How Much Evidence Do We Require?

A threshold chosen before analyzing results to decide how much evidence is required against H₀.

Collect Data → Choose a Test → Compute the Test Statistic

Compute the Test Statistic From sample data, calculate a number that measures how far the sample result is from what H₀ predicts

Common statistics: z, t

​

Interpretation: Larger ∣test statistic∣| sample is far from H₀ |→ stronger evidence against H₀

Example: Compare two means → compute a t-statistic

Choose a Statistical Test Choose based on the research question, data type, and study design:

• Compare 2 means → t-test
• Compare 3+ means → ANOVA
• Compare proportions → Chi-square/Proportion test
• Relationship between variables → Correlation / Regression

​

After computing the test statistic, decide whether to reject H₀. Two equivalent ways to make a decision:

LATER TOPIC Decision Methods in Hypothesis Testing

LATER TOPIC Decision Methods in Hypothesis Testing

How Confident Are You With Your Decision?

Confidence Level (C) & Significance Level (α)

How confident are we in our decision?

Confidence Level (C)

• Degree of certainty in our decision
• Common values: 95%, 99%
• Higher confidence → more certainty in rejecting H₀

Relationship: α = 1 − C

Example: 95% confidence → α = 0.05

Interpretation

• Higher confidence → stricter test → harder to reject H₀
• Lower confidence → easier to reject H₀

Type I and Type II Errors

Common mistakes we can make in hypothesis testing

Example: What is Type I and II error?

Scenario: Let's say you're testing whether a new drug is effective in reducing blood pressure.

Collect data from a sample of patients and conduct a statistical test.