Asking questions

Tolga Tezcan, PhD

Learning outcomes

1. Identify four different types of administering surveys (face to face, phone, mail, and online)
2. Learn behavior, attitude, and attribute survey questions
3. Identify the principles of question design (reliability and validity)
4. Learn question types (closed-ended and open-ended)
5. Learn the principles of wording questions (short, single-barreled, and neutral)
6. Learn mean, standard deviation, and normal distribution

2

Discussion question (1)

Consider the last few surveys you've taken.

How could the overall process be improved?

Was it the wording of the questions, the types of questions asked, or the surveyor's approach?

​

3

Administering Questionnaires

4

FACE-TO-FACE SURVEY

• Longer questionnaire
• Higher response rate
• Costly

PHONE SURVEY

• Rapid data collection
• Quality control
• Sampling problems

​

MAIL SURVEY

• Large sample size
• No interviewer effect
• No opportunity to clarify questions

ONLINE SURVEY

• Reduced cost
• Automatic data entry
• Lower response rate

1

2

3

4

Question content

5

Global Warming/Renewable Energy Survey - 2008 (ICPSR 36368)

European Social Survey – 2014 (ICPSR 24190)

New Family Structures Study – 2012 (ICPSR 34392)

Principles of question design

6

Consistency

Content

Question types

7

Closed-ended

Answers are provided from which respondents are to select one or more responses.

​

Latino National Survey - 2006 (ICPSR 20862)

Easier to answer

Requires less effort and less facility with words

Easier to analyze

Enhances standardization and comparability

Forcing and manipulating the respondents

Forces to give simple responses to complex issues

Open-ended

Respondents answer in their own words.

Latino National Survey - 2006 (ICPSR 20862)

Explores a topic in depth

Yields responses the designer may not have contemplated

Builds rapport

Increases the level of rapport between respondent and researcher

Coding problem

Time consuming

Introduces the variability in the coding of answers

Question types - “Other” option

8

Latino National Survey - 2006 (ICPSR 20862)

Question types - Data entry errors and “Other” option issues

9

Closed-ended

Open-ended

Wording questions (1)

10

Do you believe that poverty and inequality are caused by economic policies and lack of access to education and healthcare?

LONG?

DOUBLE-

BARRELED?

On a scale of 1 to 10, with 1 being not concerned at all and 10 being extremely concerned, how concerned are you about the wide-ranging and multifaceted impact of climate change on the environment, including but not limited to its effects on biodiversity, natural resources, ecosystems, weather patterns, sea level rise, and overall planetary health, considering the potential implications for current and future generations, and the need for urgent action to mitigate and adapt to these impacts?

LEADING?

Don't you agree that stricter gun control laws are necessary to reduce gun violence in our communities?

Wording questions (2)

11

How important to you is addressing prescription drug and other drug abuse in Maine and the recent surge in overdose deaths: very important, somewhat important, not too important, or not important at all?

​

LONG?

DOUBLE-

BARRELED?

If someone is caught with a small amount of illegal drugs for personal use, in your opinion, what should happen to them: They should be evaluated for drug issues, offered treatment but not be arrested or face jail time; they should spend no more than 3 months in jail; they should spend up to 1 year in jail; they should spend up to 3 years in jail; they should spend up to 5 years in jail; or they should spend 10 or more years in jail?

LEADING?

There are more than 2 million Americans in prison, more people by far than any other country. How important to you is substantially reducing the number of Americans in prison: very important, somewhat important, not too important, or not important at all?

Drug Policy Alliance, 2016. Public Policy Polling (Link)

Response formats

12

Do you agree that people should recycle?

​

1. Yes
2. No

Do you agree that people should recycle?

​

1. Strongly disagree
2. Mostly disagree
3. Somewhat disagree
4. Neither agree or disagree
5. Somewhat agree
6. Mostly agree
7. Strongly agree

Mean (1)

The mean is the sum of the values divided by the number of observations.

Example:

​

We asked 12 people how many burgers they can eat maximum.

​

Sample 1 (6 people): 0, 4, 4, 5, 7, 10

​

Sample 2 (6 people): 5, 6, 5, 7, 3, 4

13

Sample 1 Mean = ?

​

​

Sample 2 Mean = ?

Mean (2)

The mean is the sum of the values divided by the number of observations.

Example:

​

We asked 12 people how many burgers they can eat maximum.

​

Sample 1 (6 people): 0, 4, 4, 5, 7, 10

​

Sample 2 (6 people): 5, 6, 5, 7, 3, 4

14

Sample 1 Mean = 5

​

​

Sample 2 Mean = 5

Mean (3)

15

Sample 1 (6 people): 0, 4, 4, 5, 7, 10

Sample 1 Mean = 5

Sample 2 (6 people): 5, 6, 5, 7, 3, 4

Sample 2 Mean = 5

Standard deviation (1)

The standard deviation is a measure that indicates how much data scatter around the mean.

16

Standard deviation (2)

The standard deviation is a measure that indicates how much data scatter around the mean.

17

Sample 1 (6 people): 0, 4, 4, 5, 7, 10

Standard deviation (3)

18

Sample 1 (6 people): 0, 4, 4, 5, 7, 10

Each participant’s difference from the mean:

Normal distribution (1)

• Many naturally-occurring phenomena appear to be normally-distributed. Take, for example, the distribution of the heights of human beings. The average height is found to be roughly 175 cm (5' 9"), counting both males and females.
• As the chart below shows, most people conform to that average. Meanwhile, taller and shorter people exist, but with decreasing frequency in the population. According to the empirical rule, 99.7% of all people will fall with +/- three standard deviations of the mean, or between 154 cm (5' 0") and 196 cm (6' 5"). Those taller and shorter than this would be quite rare (just 0.15% of the population each).

19

Quincunx

Normal distribution (2)

In normal distributions,

• 68% of all data points fall within one standard deviation of the mean,
• 95% of all data points fall within two standard deviations of the mean,
• 99.7% of all data points fall within three standard deviations of the mean.

20

Normal distribution and standard deviation

21

Sample 1 (6 people): 0, 4, 4, 5, 7, 10

Statistics of rolling dice (1)

22

Click Here

Statistics of rolling dice (2)

23

Galton board

24

Click here