1 of 18

Introduce to Hypothesis Testing

CMSC 320

2026

Fardina F Alam

(PART01)

2 of 18

Topics we will cover

CMSC320 Textbook Chapter: Chapter 8

What is Hypothesis Testing
Null and Alternative Hypothesis
Level of Significance 𝛂
Collect Data using Different Sampling Methods
Type I and II Error

3 of 18

“Data alone is not interesting.

It is the interpretation of the data that we are really interested in.”

Hypothesis Testing (Making Informed Decisions with Data)

4 of 18

Hypothesis Testing

Hypothesis testing is a statistical method used to make informed decisions or draw conclusions about a population based on a sample of data.

RECAP:

Population: The entire group that you are interested in studying.

Sample: A subset of the population used for analysis.

5 of 18

Main Steps of Hypothesis Testing

State the Null Hypothesis
State the Alternative Hypothesis
Pick a Level of Significance 𝛂
Choose a Test
Collect Data
Calculate a test statistic
Calculate P-Value and compare with 𝛂
Draw a Conclusion

6 of 18

Collect Data

Obtain a representative sample from the population.

Remember the importance of recognizing whether data is collected through an experimental design or observational study.

7 of 18

Sampling Method

Random sampling� • Every unit has equal chance of selection.

A sampling method is a process by which individual items/event (observational units) are selected from the population to be included in the sample. Common sampling methods:

Stratified sampling� • Population divided into meaningful groups (strata).� • Random samples taken from each group.

Cluster sampling� • Population divided into clusters (e.g., locations).� • Randomly select clusters and include all units inside.

Systematic sampling� • Random start → select every kth unit.

Convenience sampling (non-probability)� • Select easiest or most available participants.� • Fast but may introduce bias.

8 of 18

Sampling Methods: Examples

Random sampling: Choosing names randomly from a list for a survey
Stratified sampling: Surveying transportation preferences involves dividing the population into three income strata—low, middle, and high-income households—and then randomly selecting households from each stratum in a city.
Cluster sampling: Estimating the average income of households in a city by dividing the city into clusters based on neighborhoods or districts, then randomly selecting specific neighborhoods as clusters and surveying all households within the selected neighborhoods.
Systematic sampling: Selecting every 10th person from a list of customers
Convenience sampling: Surveying people in a shopping mall (ease of access, availability), Recruiting volunteers from a specific organization or community etc.

9 of 18

Two hypotheses involved in hypothesis testing

Hypothesis: a claim we test using data (about a population).

Null hypothesis (H₀): no effect / no difference (current assumption)�Alternative hypothesis (Hₐ): real effect / difference (opposite of H₀)

Possible Outcomes of a Test

Reject H₀ → Evidence suggests a real effect or difference (support Hₐ)
Fail to reject H₀ → Not enough evidence to support Hₐ

(We never “prove” H₀ true.)

Example:� H₀: average stay = 5 days� Hₐ: average stay ≠ 5 days

Let’s say, It is believed that a candy making machine makes chocolate bars that are on average 5 gram in weight. A worker claims that the machine after maintenance no longer makes 5 gram bar. Write down H₀ and H_a.

H₀ and H_a. are mathematical opposite (mutually exclusive)

Practice: Doctors believe that the average teen sleeps on average of no longer than 10 hours per day. A researcher believes that teens on average sleep longer. Write down H0 and Ha.

10 of 18

Significance Level (α) & Decision

Significance level (α): Probability of rejecting a true null hypothesis (Type I error)

Common choices:� • α = 0.05� • α = 0.01

Set before collecting or analyzing data

Smaller α → stronger evidence required to reject H₀

Before we look at the data, we decide how much evidence we need to reject the null hypothesis.

Evidence = how different the sample data is from what the null hypothesis predicts.

This threshold is called the significance level, written as alpha (α).
Alpha represents the risk we are willing to take of being wrong when rejecting the null hypothesis.�
Specifically, it is the probability of making a Type I error — rejecting H₀ when it is actually true.�
Common choices are 0.05 or 0.01.� That means we accept a 5% or 1% chance of a false alarm.�
Smaller alpha means we require stronger evidence to reject the null hypothesis.�

Key idea:� Alpha is set before analyzing the data — it defines how strict our decision rule will be.

We measure this using:

✅ Test statistic (z, t, etc.) — how far the result is from H₀� ✅ p-value — how unlikely the result is if H₀ were true

The p-value measures how surprising the data is if the null hypothesis were true, and we reject the null if that probability is small enough.

11 of 18

Significance Level: Example

Let's say you're testing a new treatment for headaches.

Your null hypothesis (H0) : the treatment has no effect, and

Your alternative hypothesis (H1) : the treatment reduces headaches.

You set your significance level (α) at 0.05, meaning you're willing to accept a 5% chance of mistakenly concluding that the treatment works when it actually doesn't.

After conducting your study, let’s say you calculate a p-value of 0.03.

Since the p-value is less than α (0.03 < 0.05), you reject the null hypothesis and
indicates that the observed data is statistically significant, providing enough evidence to support the alternative hypothesis.

conclude that the treatment likely reduces headaches.

p-value tells us how unusual our result is if the null hypothesis is true.� It shows whether the data looks normal or strange under H₀.

p-value = chance of getting this result if nothing actually changed.

he machine is supposed to make bars with mean 5 grams (H₀).

You measure and get 5.8 grams.

Now imagine:

If the true average really is 5 grams,�
how likely is it to randomly get 5.8 from sampling?�

If that chance is very small → p-value is small → result is unusual → reject H₀.

Suppose a candy machine is supposed to make bars that weigh 5 grams on average (this is H₀).

You test some bars and get an average of 5.8 grams.

If this kind of result is very unlikely when the true average is 5 grams → small p-value → result looks strange → reject H₀�
If this kind of result is fairly common when the true average is 5 grams → large p-value → result looks normal → keep H₀�

⭐ One-line takeaway

If the result would rarely happen when H₀ is true → we doubt H₀.

12 of 18

Collect Data, Choose a Statistical Test & Compute Test Statistic

Choose a Statistical Test Based on research question, data type, and number of groups:

Compare 2 means → t-test
Compare >2 means → ANOVA
Compare proportions → Chi-square test
Relationship between variables → Correlation / Regression

Compute the Test Statistic From sample data, calculate a number that measures how far the sample result is from what H₀ predicts

Common statistics: z, t

Interpretation: Large magnitude → sample is far from H₀ → stronger more evidence against H₀

Example: Compare two means → compute a t-value

13 of 18

Decision Methods in Hypothesis Testing

After computing the test statistic, decide whether to reject H₀. Two equivalent ways to make a decision:

1. Critical Value Approach

Compare test statistic to critical value (cutoff).

2. p-value Approach

Compare p-value to significance level α.

Both methods give the same conclusion.

Decision: Is the Result Extreme Enough?

14 of 18

(LATER TOPIC) Decision Methods in Hypothesis Testing

Rejection Region (Critical Value Approach)

Values that are too extreme if H₀ were true.
Test statistic falls in

rejection region → Reject H₀
Otherwise → Fail to reject H₀

P-value Approach

Probability of observing results this extreme if H₀ is true.

p ≤ α → Reject H₀
p > α → Fail to reject H₀

Key Idea

More extreme result → stronger evidence against H₀

15 of 18

Q: How confident you are with your decision?

Ho: μ = 5 gram. ; Ha : μ ≠ 5 gram.

Possible Outcomes of this test: Reject the null hypothesis or Fail to Reject the null hypothesis

Example: We sample 50 chocolate bars and get average value of mass of the bar.

Then we calculate test statistics (depends on what type of problem you have)
It help to determine that the data you have statistically significant enough to reject this null hypothesis or not.

Let’s say:

Monday → Received average value of mass of the bar 5.12 gram

Wednesday → Received average value of mass of the bar 5.75 gram

Friday → Received average value of mass of the bar 7.82 gram

How much I am confident that I should reject the Ho or not! → we need a CONCRETE WAY to look at the Ho

16 of 18

Confidence Level (C) & Significance Level (α)

How confident are we in our decision?

Remember: Significance Level (α): risk of being wrong (rejecting a true H₀)

Confidence Level (C)

Degree of certainty in our decision
Common values: 95%, 99%
Higher confidence → more certainty in rejecting H₀

Relationship: α=1−C

Example: 95% confidence → α = 0.05

Interpretation

Higher confidence → stricter test → harder to reject H₀
Lower confidence → easier to reject H₀

How to decide C or Alpha value? depends on several factors, including the nature of the research, the consequences of making Type I and Type II errors, and the standard practices in the field. A general guide for choosing the value:

17 of 18

Type I and Type II Errors

Mistakes we can make in hypothesis testing:

Type I Error (False Positive) Reject H₀ when it is actually true

Think something changed, but it didn’t
Probability = α
Type I = false alarm

Type II Error (False Negative) Fail to reject H₀ when it is actually false

Miss a real change
Type II = missed detection

These errors are important to consider because they affect the validity of the conclusions drawn from hypothesis testing.

Practice: A company stated that their straw machine makes straws that are 4mm in diameter. But a worker believes that the machine no longer makes straws of this size and samples 100 straws to perform a hypothesis test with 99% confidence . Write down H0, H, N, C, alpha.

18 of 18

Example: What is Type I and II error?

Collect data from a sample of patients and conduct a statistical test.

Let's say you're testing whether a new drug is effective in reducing blood pressure.

Your null hypothesis (H0) : the drug has no effect on blood pressure, and

Your alternative hypothesis (H1) : the drug reduces blood pressure on avg.

Type I error → concluding that the drug is effective at reducing blood pressure when, in reality, it has no effect. (a false positive result, indicating the drug works when it doesn't.)

Type II error → failing to detect that the drug is effective at reducing blood pressure when, in fact, it does reduce blood pressure (a false negative result, indicating the drug doesn't work when it actually does.)