Methods to Analyze Surveys: Continuous Quantitative Data

(Analyzing and Presenting

Pre-Post Evaluation Survey Data)��Sept. 23, 2025

​

Welcome!

​

While we wait for everyone to join…

​

Check Your Zoom Name

• On your square-right corner-click blue box

• Select “Rename”

• Enter full first and last name (if not already listed)

​

​

Kit Alviz, Evaluation Analyst, Program Planning and Evaluation

Samuel Ikendi, Academic Coordinator, Climate Smart Agriculture

​

​

​

Anticipated Outcomes

• Understanding of different measures of central tendency and variability
• Hands-on experience analyzing and reporting frequency distribution in pre-post tests using Excel
• Hands-on experience running a paired-samples t test in Excel, interpreting the results, and reporting the findings
• Understanding of levels of quantitative data

​

Training Outline

• 10:00 - Summary statistics
• 10:10 - Scales of Measurement
• 10:22 - Methods to run Frequency Distributions from Pre- and Post-tests
• Activity: Point-Change Frequency Distribution Method – Computing, Interpreting and Reporting
• 11:05 - 5-minute break
• 11:10 - T-tests
• Activity: Paired t-test - Computing, Interpreting and Reporting

Source: https://media.geeksforgeeks.org/wp-content/uploads/20250222144920592095/stat.webp

Descriptive Statistics

Measures of Central Tendency

• Index representing a Center point or Central location

• Three common measures

Central Tendency contd…

• Mean or arithmetic average: Most used measure

​

​

e.g.: Observations: 14 15 16 17 18.

Total observations = 5

Total sum of observations (14+15+16+17+18) = 80

Mean = 80/5 = 16

Central Tendency contd…

• Median: Point in the distribution below which 50 percent of the cases lie.

e.g.: 1) 14 15 16 17 18 = 16. e.g.: 2) 14 15 16 17 18 19: Median is (16+17)/2 = 16.5

​

• Two examples

Central Tendency contd…

• Mode: Value that occurs most frequently.

Unimodal: 14 16 16 17 18 19 19 19 21 21 22

Bimodal: 14 16 16 16 17 18 19 19 19 21 21 22

Multimodal: 14 16 16 16 17 18 19 19 19 21 21 22 22 22 23

Measures of Variability/Dispersion

Index representing Spread, Dispersion or Distribution.

Example: the mean values of two distributions may be identical,

• whereas the variability may be different.

E.g.: A) 24, 24, 25, 25, 25, 26, 26 : Mean = 25; B) 10, 15, 22, 28, 31, 34, 35 : Mean = 25

• The mean in both the distributions is 25, but
• the degree of scattering of the scores differs considerably.

Indices of Variability

Range

​

E.g.: 2 10 11 12 13 14 15 16 : Range is 16-2 = 14

Interquartile Range: Range between 75 percentile (Q3) and 25 percentile (Q1).

Standard Deviation and Variance

Standard deviation

• is a measure of variation in a dataset. How close are you to the mean

Variance

• measures the spread between numbers in a dataset.

Frequency Distribution and Outliers

Counting how many times each category is occurs.

• Presented by histogram, freq. polygons, error bar, box plot.

​

• Outlier: Observation that is far away from normal
• The outliers can be identified by looking at boxplots, scatterplots, and histogram (alternative: calculate mean, median, sort column to see highs and lows)

Source: https://datapeaker.com/en/big--data/how-to-detect-and-eliminate-outliers/

Scales of Measurement

• Measurement: The process through which observations are translated into numbers.
• Measurement is the “assignment of numerals to objects or events according to rules” (Stevens, 1951, p.1).
• Steven’s Scales of Measurement:
• Nominal
• Ordinal
• Interval
• Ratio

Nominal

• Categories are qualitatively different, NOT quantitatively.
• Two or more relevant, mutually exclusive categories.
• The only relationship between the categories is that they are different from each other; one category does not represent ”more” or ”less” of the characteristic/variable.
• Examples: School district number, race/ethnicity, party affiliation, marital status.
• Changing assigned number codes does not make any difference for statistical analysis.

​

Ordinal

• Ranks objects with respect to how much or how little of the attribute they possess.
• Data are classified into categories that can be ordered or ranked, but without indicating the degree of difference between them.
• Examples: football rankings, guilty/not guilty when making judgments, opinion scales, �self-reported knowledge* �or behavior change scales

​

Interval

• Not only orders objects or events but has equal intervals between the units.
• Equal differences in the numbers represent equal differences.
• Examples: Celsius temperature, IQs, educational tests.
• There is no absolute zero, meaning zero doesn’t indicate the absence of the variable attribute being measured.
• Can’t say that a student receiving zero on a statistics test has zero knowledge of statistics.

Ratio

• Has equal intervals as well as true zero point.
• A yardstick to measure length in units of inches or feet where the origin on the scale is an absolute zero corresponding to no length at all.
• Examples: work experience, farm income, milk production, test weight.

1. Do you have non-farm income sources?

Yes    - No   - Prefer not to say 

Nominal

2. What was your annual farm income in dollars last year?

$40,000-$50,000    - $50,001-$60,000    - $60,001-$70,000    - >$70,000 

Ordinal

3. What is your overall grade point average in your undergraduate program? Interval

4. What was your annual farm income in dollars last year? Ratio ���

​

​

Scales of Measurement Recap

​

​

​

​

Point-Change Data Analysis Example

​

​

Methods to Analyze Pre-post Data Using Frequency Distributions

1. Change in the top two categories of the scale (high + very high)

​

Frequency distribution of campers based on self-reported knowledge levels before and after attending the camp

Likert-type scale categories. 1: No Understanding, 2: Slight Understanding, 3= Moderate Understanding, 4= Good Understanding, 5= Excellent Understanding.

Methods to Analyze Pre-post Data Using Frequency Distributions

2. Individual point-change

​

​

​

Frequency Distribution of Learners Based on their Understanding of Soil Health Concepts Taught in the Course (n= 13)

​

Q1. Understand how the physical characteristics of soil contribute to soil health

Likert-type scale categories. 1: No Understanding, 2: Slight Understanding, 3= Moderate Understanding, 4= Good Understanding, 5= Excellent Understanding.

Individual Point-Change in Excel

1. Calculate Q1 Change in Sheet 1 (B - A)
2. Calculate frequencies in column I using COUNTIF
1. Click on function key fx
2. The ”Insert Function” window will pop up.
3. Select ‘COUNTIF’ and click OK
4. In the next window, you will be asked to choose a “Range” and “Criteria”.
5. Range: the group of cells that contain the data you want to analyze. Tip: Make range an absolute reference $B$4:$B$33 for efficiency
6. Criteria: Click the cell of the specific response item for which you want frequencies.
7. Percent: Divide frequency of each response item by the total number of responses, hit the % button in the menu bar.
3. Repeat for each Point Change
4. Fill out the table “Frequency Distribution…”
5. Write a sentence highlighting a finding for reporting

​

​

Point-Change Data Analysis Activity

(30 minutes)

​

​

Group Discussion: Reporting example

BREAK

​

Inferential Statistics

• Infer from a sample to population
• Tests of significance; Involves testing of null hypothesis
• A statement that there is no statistically significant relationship between the variables and that any observed relationship is only a function of chance.

Source: https://www.collidu.com/presentation-inferential-statistics

Assumptions

• Normality
• For t-tests we assume sampling distribution is normally distributed
• Homogeneity of variance
• Variance should be same throughout the data
• Interval data
• Data should be measured at least at the interval level
• Independence
• Differs depending upon the tests
• Data from different participants should be independent

t-Tests

• Compare mean scores between two groups for statistically significant difference.

Two types: Independent samples and Dependent or paired samples

Independent Samples t-Test

• Based on two independently drawn samples.
• Allows a comparison of the means of two groups – presumably after a treatment has occurred.

Example

• Comparing means of test scores on two sampled groups, one taught traditionally and the other taught through distance education.

Dependent or Paired Samples t-Test

• Used to compare mean scores obtained from the same group.

Example:

• pre-post test score means

Type I and Type II Errors

• Type I error: Rejecting a true null hypothesis.
• Type II error: Retaining a false null hypothesis.
• Type I error (α): False alarm: The investigator thinks there is something when there is nothing there.
• Type II error (β): Miss: The investigator concludes there is nothing when there really is something.

Level of Significance

• A predetermined level at which a null hypothesis would be rejected.
• p of 0.05 = the estimated probability of the observed relationship being caused by error/chance is 5 in 100 or less.
• p of 0.01= the estimated probability of the observed relationship being caused by error/chance/ is 1 in 100 or less.

​

​

Paired t-Test Data Analysis Activity

(screenshare)

Paired Samples t-test in Excel

1. Under Data menu select Data Analysis
2. Choose “t-Test: Paired Two Sample for Means”. OK.
3. Specify range of cells containing the data for variable 1. Manually enter or click the first red, white, & blue icon, then highlight your first column of cells, including its heading. Enter. Repeat for variable 2. Enter.
4. Check the Labels box, so Excel knows you included headings atop each column. OK.
5. Excel whips out an Output table. There’s lots of info there, but all you’re really after are those p-values.
6. Calculate standard deviation for both variables =sqrt(B5)
7. In the Reporting Examples sheet, fill in all the colored cells.
8. Write one sentence describing the finding for reporting.

​

​

​

Paired t-Test Data Analysis Activity

(30 Minutes)

​

​

Group Discussion: Reporting example

Combined Reporting Example

IPM In-Service Evaluation Training participants responded to a post-event survey (n=30)

and reported a statistically significant difference (p<0.05) between self-reported understanding of how to use program logic models before the training (average 3.13 out of 5) and after the training (average 3.9 out of 5). About two-thirds (66%) of the participants reported increased understanding in how to use a logic model in program evaluation.

Feedback and Q&A

​

​

Quantitative Data Analysis Resources

• Newberry, III, O’Leary, & Israel (2017). The Savvy Survey #16: Data Analysis and Survey Results. UF/IFAS Extension, University of Florida.
• CGBS M&E Science. Student Research. How to run statistical tests in Excel.
• Leahy, J. (2004). Using Excel for analyzing survey questionnaires. Program Development and Evaluation. University of Wisconsin Extension.
• Taylor-Powell, E. (1989). Analyzing quantitative data. Program Development and Evaluation. University of Wisconsin Extension.
• Koundinya, V. (2017). How to compute and present individual change with before and after survey data. Program Development and Evaluation. University of Wisconsin Extension. https://cdn.shopify.com/s/files/1/0145/8808/4272/files/G4152.pdf
• IGIS’ 4 R trainings:Data Wrangling I & II, Reporting with Quarto, and working with spatial data: https://ucanr.edu/program/informatics-and-gis-program/collection/training

References

• Ary, D., Jacobs, L. C., Irvine, C. S., & Walker, D. (2019). Introduction to research in education (10th ed).

​

• Ary, D., Jacobs, L. C., Razavieh, A., & Irvine, C. S. (2006). Introduction to research in education (7th ed).

​

• Davis, J. A. (1971). Elementary survey analysis. Englewood, NJ: Prentice Hall.

​

• Miller, G. S. (2006). Introduction to research. AgEdS510 Course Materials.

​

• National Institute of Standards and Technology. What are outliers in the data? https://www.itl.nist.gov/div898/handbook/prc/section1/prc16.htm

• Stevens, S. S. (1951). Mathematics, measurement, and psychophysics. In S. S. Stevens (Ed.), Handbook of experimental psychology. New York: Wiley.

​

​