When you want somebody's opinion, you ask for it. Right? That's easy enough when you're just dealing with one or a few people. When you want somebody's opinion, you ask for it. Right? That's easy enough when you're just dealing with one or a few people. But what if you want to know the opinion of an entire town or an entire population? Getting an answer out of everyone in your town or every member of a particular group is nearly impossible. So how do you get an idea of what these folks think? You use a survey.
Conducting surveys can be done very simply, or it can be very complicated, depending on how much you want to ask on the survey and the number of people to whom it is administered. This section will mainly focus on doing surveys on a fairly small local scale, and we will give you some ideas about where to find information should you need to do a survey on a larger scale.
What are surveys?
A survey is a way of collecting information that you hope represents the views of the whole community or group in which you are interested.
There are three main ways of going about this:
Surveys are usually written, although sometimes the surveyor reads the questions aloud and writes down the answers for another person; they can be distributed by mail, fax, e-mail, through a web page, or the questions can be asked over the phone or in person.
Surveys collect information in as uniform a manner as possible -- asking each respondent the same questions in the same way so as to insure that the answers are most influenced by the respondents' experiences, not due to how the interviewer words the questions.
Why should you conduct a survey?
You can collect information about the behaviors, needs, and opinions using surveys. Surveys can be used to find out attitudes and reactions, to measure client satisfaction, to gauge opinions about various issues, and to add credibility to your research. Surveys are a primary source of information -- that is, you directly ask someone for a response to a question, rather than using any secondary sources like written records.
You can use surveys to measure ideas or opinions about community issues related to your initiative. For example, you may want to know how many people use your services, what users think about your services, what new users expect from your services, and whether users are satisfied with what you provide.
Deciding whether to conduct a survey
There are advantages in doing surveys, but you should consider whether a survey will be the best way of obtaining the information you need. Even though surveys are a useful method of gathering information, they are not the only way. You will need to decide whether a survey will produce the information you need. The information you need may be obtained through other means, such as informal unstructured conversation that takes place in the course of another activity; census figures; meeting with people in the community; interviews; or observation.
When should you conduct a survey?
A survey may be your best choice when:
Written surveys: Pros and Cons
Advantages of written surveys:
Disadvantages of written surveys:
If you have decided that what you need is a large-scale, formal survey, hiring someone to do it for you or working with local colleagues or a nearby university may be your best bet. If you're going to do it on your own, keep in mind that some people you present your report to may not give much credit to a survey you did on your own.
How do you prepare a survey?
Decide on the purpose of the survey.
If you have decided to do a survey, you must first be sure exactly why you're doing it. What questions do you want to answer? Is it to get a general idea of the demographics of your area? To find out what people think about a particular issue or idea? Or is there another reason you're considering a survey?
In any case, you will need to keep the purpose of the survey in mind throughout the process, as it will influence the choice of questions, the survey population, and even the way the survey is delivered (e.g., a computer-savvy population can be surveyed over the Internet; a population that is largely illiterate shouldn't be asked to take a written survey, and so forth).
Example: 1997 Youth Risk Behavioral Survey purpose
The Youth Risk Behavioral Survey (or YRBS) is done annually by the Centers for Disease Control to identify behaviors that pose health risks among young people in America. We will be using the 1997 and 1999 YRBS for examples in this section.
The CDC decided its purpose in this survey was to track the health risk behaviors that cause the most deaths among youth. Also, many of those behaviors are included in the survey because they begin in youth and continue into adulthood, having significant impact on adult health later on. Here are some of the behaviors the YRBS attempts to measure:
Decide whom you will survey.
The next step is finding out who has the answers to your question or questions. In other words, it's time for you to determine your audience -- the people who can best answer the questions your initiative needs to ask. Who will you survey? Is it the general public? The current program beneficiaries? People in a specific neighborhood or segment of the community? Potential members?
Sampling
Almost all surveys rely on sampling -- that is, identifying a section of your population that satisfies the characteristics you're trying to survey, rather than trying to do a census.
To have a truly representative sample, you must be sure that every member of the group you want to survey has an equal chance of being in the sample, and/or you must have a fairly large sample. It's important to make sure that the sample size you choose is adequate and not excessively large or small. If too large, it may be impossible to survey everybody effectively and within your budget; if too small, your credibility may suffer. A general rule to keep in mind is that the larger the sample size, the more accurate a reflection of the whole it will be.
You can figure out how big your sample should be by using a sample size calculator, such as:
Sample Design Issues
You might also need to give some thought to the design of your sample, especially if you are hoping to get representative responses from two or more groups. For example, let's say you are doing a survey on youth violence and you want to get responses from youth, parents, and educators; this means that you'll need to come up with separate population counts for each of these groups and then select a sample from each. The samples should be large enough to represent the group it is drawn from, but the sample sizes should be proportional to the groups they represent.
For example, you might design a sample that comes out like this:
| Youth | Parents | Educators |
Population | 650 | 200 | 500 |
Sample | 65 | 20 | 50 |
Sampling can be a complex topic; before you start, you might want to learn some of the basic terminology and concepts from resources such as Sampling Terminology and Survey Sample Sizes and Margins of Error.
Potential pitfalls
Sampling is a challenge to conducting good surveys, but there are other pitfalls. For example, when people volunteer to respond to a survey, we say they are self-selected. These people may have a special interest in answering your survey, so their answers may not be truly representative of the group you're interested in. There are ways of dealing with self-selected audiences, such as only using a random selection of their surveys when only self-selection is involved.
For example, if you get back 300 completed surveys, you might decide to only use every third one in order to randomize the results.
Decide what method you will use to collect your survey data.
Will your survey be written or oral? Is there going to be a number where people can call to register their results? Are you going to have a post office box to which completed surveys should be mailed? You need to decide whether it's going to be administered by people known to the audience and whether it will be done in person, by phone, or by mail. Remember that the more personal you make it, the higher the return rate will be. Surveys that are delivered cold have a return rate of only two to three percent, unless they're on a very hot topic for the community you're surveying.
Keep in mind whom you want to survey. Does your public feel more comfortable writing or speaking? Will it be efficient to leave surveys somewhere for people to pick up at their will, or should you do something to make sure they get one? If your survey is to be administered orally, will people feel honored or annoyed about being asked for their opinions?
Mailed questionnaires are a very useful tool in your information-gathering bag of tools. It's a much cheaper alternative to other types of information gathering and it allows you to get information from many people across long distances without paying extremely high phone bills. If you're considering doing a mailed survey, be sure to check with your local post office for information on mailing regulations, bulk mail rates, and so on.
Some advantages of mailed questionnaires are:
Some disadvantages of mailed questionnaires are:
How long should your survey be?
When determining the length of your survey, remember that less is more. The longer it is, the less likely it is that people will take the time to do it. People get bored with long surveys, and usually won't even bother to look at a survey that is more than a page and a half long. Also, requiring long answers may lose your audience. Through editing and condensing, you should try to keep your survey down to one page.
What it is you want to know and the method of survey (e.g., phone survey, mailed survey) will also influence the length of your survey. Phone surveys, for example, can take a little longer to complete.
Once you've decided on your method, you can go on to write your questions. We'll talk in more detail about distributing your survey later on.
Example: 1997 Youth Risk Behavioral Survey sampling
The 1997 YRBS used a type of sampling called cluster sampling. In cluster sampling, the entire population is divided into groups, or clusters, and a random sample of these clusters are selected. For example, age group or geographical location determined the YRBS's clusters. All observations in the selected clusters are included in the sample. This technique is used in large-scale surveys where it may be more convenient to sample clusters than to do a pure random sample.
Write your questions.
When preparing the questions, bear in mind that they can take many forms. Questions might be:
For example: "Why is it important to use condoms?" These questions are used when you want to find out what leads people to specific behaviors, what their attitudes are towards different things, or how much they know about a given topic; they provide good anecdotal evidence. The drawback to using open-ended questions is that it's hard to compile their results.
For example: "Do you use condoms?" These are used when the information you need is fairly clear-cut, i.e., if you need to know whether people use a particular service or have ever heard of a specific local resource.
For example: "When I have sex, I use condoms... a) every time, b) most times, c) sometimes, d) rarely, e) never." These allow you to find out more detailed information than closed-ended questions, and the results can be compiled more easily than open-ended questions.
1 = strongly disagree
2 = disagree
3 = undecided
4 = agree
5 = strongly agree
If you want to weed out neutral and undecided responses you can use an even-numbered scale with no middle "neutral" or "undecided" choice. In this situation, the respondent is forced to decide whether he or she leans more towards the "agree" or "disagree" end of the scale for each item. The final score for the respondent on the scale might be the sum of his or her ratings for all of the items.
Example: Using the Likert scale
Here are a few sample survey questions in Likert scale format, done without a neutral category:
Please check the answer indicating your reaction to the questions listed below.
The questions you ask depend on the audience you're trying to reach and the information you're trying to obtain.
For example, for demographic information (e.g., questions that determine where people are from, their ages, and their incomes), you should make the survey all check-offs, yes/no questions, and fill-in-the-blank questions so that it?s as easy as possible to complete.
Creating surveys people will answer with the Total Design Method
Low response rates are a major problem with surveys; it's common for the response rate to be as low as 30%. One way of avoiding low response rate is to use the Total Design Method, which was developed by Don Dillman of Washington State University. Dillman's method has been shown to yield an average return rate of 73%.
Questions should be worded carefully in order to yield exactly the information you're looking for.
To make sure your survey works the way you want it to, try it out on a few members of the population you're aiming at before you actually distribute it.
Guidelines for writing your survey questions:
Example: 1999 Youth Risk Behavioral Survey questions
Here are a few examples of questions from the 1999 YRBS.
A. 0 times
B. 1 time
C. 2 or 3 times
D. 4 or 5 times
E. 6 or more times
A. Yes
B. No
A. I have never smoked a whole cigarette
B. 8 years old or younger
C. 9 or 10 years old
D. 11 or 12 years old
E. 13 or 14 years old
F. 15 or 16 years old
G. 17 years old or older
A. 0 times
B. 1 or 2 times
C. 3 to 9 times
D. 10 to 19 times
E. 20 to 39 times
F. 40 or more times
A. I have never had sexual intercourse
B. 1 person
C. 2 people
D. 3 people
E. 4 people
F. 5 people
G. 6 or more people
How do you distribute your survey?
There are several strategies for distributing surveys. We'll talk about the most common one -- direct mail -- in the most detail, but there are many methods to choose from and there is no one perfect method. You may want to use a combination of methods.
Here are a few thoughts to help you decide on your method:
For example, if you are doing a survey on sexual risk behaviors, people may be uneasy telling an interviewer how many partners they've had or other such details.
Direct mail
Direct mailing your survey to people whose addresses are known is the most common strategy. Distributing a survey by mail has a high percentage of non-responders (you're lucky if 30% respond, although it tends to be higher in small communities), but it's a lot easier than many other methods and takes less staff hours.
Interviews and phone surveys
For those who have difficulty reading or using printed materials, or for surveys that require more in-depth answers, interviews might be the most appropriate thing for you to do. Phone surveys work similarly to face-to-face interviews, so we've grouped these two methods together.
Drop boxes
Agencies that have relatively frequent contact with clients -- such as once a month -- you may find that setting up a drop box in their offices are a good source point for distributing surveys. This may also be a good option for agencies that have an incomplete mailing list. It can also be a good way to contact clients of other agencies who have little contact with your group or agency. However, if you use this method of distributing surveys, consider using it along with at least one other method of distribution, because only those already using the services can respond.
Media distribution
For general distribution, publishing a survey in the local paper or attaching a survey to your newsletter might be a good idea.
Convenience sampling
Taking surveys in a public place -- setting up a booth or table in the parking lot at a local discount store, on the sidewalk in the shopping district, etc. -- provides an opportunity to get some exposure for your organization.
Group administration
If your group or organization tends to have large group gatherings, providing surveys to everyone who attends a particular gathering might be a really efficient way for you to gather information. Examples of gatherings where you might want to distribute your survey would include: immunization clinics, commodity food distribution sites, health fairs, and meal sites for older adults. If you want to give your survey out at some sort of group meeting or gathering, get the group's director to put you on the agenda. At the meeting, introduce yourself and explain the purpose of the survey. Then distribute the survey, answer any questions, and collect completed surveys. Don't forget to thank everyone for their participation!
Using multiple methods of distribution
You can combine or adapt two or more of the above methods to suit your own purposes, if you'd like. If more than one method is used, each survey should include instructions that each citizen should complete only one survey. So, for example, if you're having people complete surveys at a booth at the county fair, they should not complete the survey if they've already completed one that came in the mail to their homes.
Collecting the surveys
Soon after the surveys are distributed, some of them will begin to arrive at the sponsoring organization. Here are the steps you should take to collect your surveys:
Example: Administering and collecting the 1997 Youth Risk Behavioral Survey
The CDC wanted to do everything it could to protect the students' privacy and insure that questions would be answered honestly while completing the YRBS. In order for the survey to be administered voluntarily and anonymously, it was done in a self -administered written questionnaire containing 84 multiple-choice questions. Before the surveys were administered, parental permission was obtained through whatever methods those local schools used. Students recorded their responses to the questionnaires on computer-scannable answer sheets, further allowing for anonymity.
How do you analyze and compile the results of your survey?
Now that you've gathered the completed surveys, you'll need to figure out the results. Sometimes all you have to do is tabulate the results -- that is, add them up and display in a table.
For instance, if 100 questionnaires were returned in a survey about problems in the neighborhood, you just need to count the answers. Let's say that there was a question asking what people felt was the biggest challenge facing the neighborhood; 70 people mentioned law enforcement, 10 cited transportation, 15 marked potholes, and 5 said noise. The result in cases like this is clear.
However, analysis can be far more complicated than that. If you're looking,
for instance, at how people feel about a service or problem, you may end up with a lot of answers to open-ended questions that are apparently unrelated. In this case, you will need to try to find patterns.
Once you've done that, what do these numbers mean? Well, you will need to look at the overall survey to see how each percentage compares to the others.
For example, what questions had the highest proportions of similar responses?
We suggest that you write up a brief report -- one page is sufficient -- summarizing the results of the survey. In your report, look for any patterns -- do people in a particular part of town feel more strongly about a particular issue than those in other areas?
Share this information with your staff. Get their feedback and discuss whether any further surveying needs to be done before completing.
Now that you've figured out what the results mean, you need to decide what to do with them. To whom are you going to communicate them, and how? In case of a community initiative, the results should be made public as soon as possible so that members in the community and community leaders can be made aware of a problem or potential problem and start working to solve it. If other similar surveys have done in the same area, you may want to compare your results with the other surveys' results.
An organization conducting a survey about its' services might want to use results to provide a better service or to change a current policy to a more efficient one. In a situation where funding is at stake, the results would need to go to the funder to convince the funder of the need for new or continued support. The results could also be used by the organization itself to determine where and what kinds of services are needed. The 2003 Youth Risk Behavior Survey results will give you an idea of what a survey result report looks like.
A well-executed survey can provide your initiative with a wealth of information about your constituents and their needs. We hope this section has given you the tools you need to conduct surveys that are effective and that give you the information you need to serve your constituents better!
Contributor
Chris Hampton
Marcelo Vilela
Online Resources
Kalsbeek, B. (1995). How to Collect Survey Data.
Kalsbeek, B. (1995). How to Plan a Survey.
Hassett, Nancy. (1997). Sample design issues. In FOCUS: An information newsletter for health care professionals, Winner 1997.
de Leeuw, J. (1997). Sample Size Calculator.
Niles, R. (1999). Statistics Every Writer Should Know.
Easton, V.J., and McColl, J.H. (1999). Statistics Glossary, v.1.1.
Print Resources
Bailey, K. (1978). Methods of social research. New York, NY: Free Press.
Berg, B. (1995). Qualitative research methods for the social sciences. Boston: Allyn and Bacon.
Fink, A. (1985). How to conduct surveys: A step-by-step guide. Beverly Hills: Sage Publications.
Community Toolbox
KU Center for Community Health and Development. (2023). Chapter 3, Section 1: Developing a Plan for Assessing Local Needs and Resources. Lawrence, KS: University of Kansas. Retrieved January 4, 2023, from the Community Tool Box: https://ctb.ku.edu/en/table-of-contents/assessment/assessing-community-needs-and-resources/develop-a-plan/main
opulation? Getting an answer out of everyone in your town or every member of a particular group is nearly impossible. So how do you get an idea of what these folks think? You use a survey.
Conducting surveys can be done very simply, or it can be very complicated, depending on how much you want to ask on the survey and the number of people to whom it is administered. This section will mainly focus on doing surveys on a fairly small local scale, and we will give you some ideas about where to find information should you need to do a survey on a larger scale.
What are surveys?
A survey is a way of collecting information that you hope represents the views of the whole community or group in which you are interested.
There are three main ways of going about this:
Surveys are usually written, although sometimes the surveyor reads the questions aloud and writes down the answers for another person; they can be distributed by mail, fax, e-mail, through a web page, or the questions can be asked over the phone or in person.
Surveys collect information in as uniform a manner as possible -- asking each respondent the same questions in the same way so as to insure that the answers are most influenced by the respondents' experiences, not due to how the interviewer words the questions.
Why should you conduct a survey?
You can collect information about the behaviors, needs, and opinions using surveys. Surveys can be used to find out attitudes and reactions, to measure client satisfaction, to gauge opinions about various issues, and to add credibility to your research. Surveys are a primary source of information -- that is, you directly ask someone for a response to a question, rather than using any secondary sources like written records.
You can use surveys to measure ideas or opinions about community issues related to your initiative. For example, you may want to know how many people use your services, what users think about your services, what new users expect from your services, and whether users are satisfied with what you provide.
Deciding whether to conduct a survey
There are advantages in doing surveys, but you should consider whether a survey will be the best way of obtaining the information you need. Even though surveys are a useful method of gathering information, they are not the only way. You will need to decide whether a survey will produce the information you need. The information you need may be obtained through other means, such as informal unstructured conversation that takes place in the course of another activity; census figures; meeting with people in the community; interviews; or observation.
When should you conduct a survey?
A survey may be your best choice when:
Written surveys: Pros and Cons
Advantages of written surveys:
Disadvantages of written surveys:
If you have decided that what you need is a large-scale, formal survey, hiring someone to do it for you or working with local colleagues or a nearby university may be your best bet. If you're going to do it on your own, keep in mind that some people you present your report to may not give much credit to a survey you did on your own.
How do you prepare a survey?
Decide on the purpose of the survey.
If you have decided to do a survey, you must first be sure exactly why you're doing it. What questions do you want to answer? Is it to get a general idea of the demographics of your area? To find out what people think about a particular issue or idea? Or is there another reason you're considering a survey?
In any case, you will need to keep the purpose of the survey in mind throughout the process, as it will influence the choice of questions, the survey population, and even the way the survey is delivered (e.g., a computer-savvy population can be surveyed over the Internet; a population that is largely illiterate shouldn't be asked to take a written survey, and so forth).
Example: 1997 Youth Risk Behavioral Survey purpose
The Youth Risk Behavioral Survey (or YRBS) is done annually by the Centers for Disease Control to identify behaviors that pose health risks among young people in America. We will be using the 1997 and 1999 YRBS for examples in this section.
The CDC decided its purpose in this survey was to track the health risk behaviors that cause the most deaths among youth. Also, many of those behaviors are included in the survey because they begin in youth and continue into adulthood, having significant impact on adult health later on. Here are some of the behaviors the YRBS attempts to measure:
Decide whom you will survey.
The next step is finding out who has the answers to your question or questions. In other words, it's time for you to determine your audience -- the people who can best answer the questions your initiative needs to ask. Who will you survey? Is it the general public? The current program beneficiaries? People in a specific neighborhood or segment of the community? Potential members?
Sampling
Almost all surveys rely on sampling -- that is, identifying a section of your population that satisfies the characteristics you're trying to survey, rather than trying to do a census.
To have a truly representative sample, you must be sure that every member of the group you want to survey has an equal chance of being in the sample, and/or you must have a fairly large sample. It's important to make sure that the sample size you choose is adequate and not excessively large or small. If too large, it may be impossible to survey everybody effectively and within your budget; if too small, your credibility may suffer. A general rule to keep in mind is that the larger the sample size, the more accurate a reflection of the whole it will be.
You can figure out how big your sample should be by using a sample size calculator, such as:
Sample Design Issues
You might also need to give some thought to the design of your sample, especially if you are hoping to get representative responses from two or more groups. For example, let's say you are doing a survey on youth violence and you want to get responses from youth, parents, and educators; this means that you'll need to come up with separate population counts for each of these groups and then select a sample from each. The samples should be large enough to represent the group it is drawn from, but the sample sizes should be proportional to the groups they represent.
For example, you might design a sample that comes out like this:
| Youth | Parents | Educators |
Population | 650 | 200 | 500 |
Sample | 65 | 20 | 50 |
Sampling can be a complex topic; before you start, you might want to learn some of the basic terminology and concepts from resources such as Sampling Terminology and Survey Sample Sizes and Margins of Error.
Potential pitfalls
Sampling is a challenge to conducting good surveys, but there are other pitfalls. For example, when people volunteer to respond to a survey, we say they are self-selected. These people may have a special interest in answering your survey, so their answers may not be truly representative of the group you're interested in. There are ways of dealing with self-selected audiences, such as only using a random selection of their surveys when only self-selection is involved.
For example, if you get back 300 completed surveys, you might decide to only use every third one in order to randomize the results.
Decide what method you will use to collect your survey data.
Will your survey be written or oral? Is there going to be a number where people can call to register their results? Are you going to have a post office box to which completed surveys should be mailed? You need to decide whether it's going to be administered by people known to the audience and whether it will be done in person, by phone, or by mail. Remember that the more personal you make it, the higher the return rate will be. Surveys that are delivered cold have a return rate of only two to three percent, unless they're on a very hot topic for the community you're surveying.
Keep in mind whom you want to survey. Does your public feel more comfortable writing or speaking? Will it be efficient to leave surveys somewhere for people to pick up at their will, or should you do something to make sure they get one? If your survey is to be administered orally, will people feel honored or annoyed about being asked for their opinions?
Mailed questionnaires are a very useful tool in your information-gathering bag of tools. It's a much cheaper alternative to other types of information gathering and it allows you to get information from many people across long distances without paying extremely high phone bills. If you're considering doing a mailed survey, be sure to check with your local post office for information on mailing regulations, bulk mail rates, and so on.
Some advantages of mailed questionnaires are:
Some disadvantages of mailed questionnaires are:
How long should your survey be?
When determining the length of your survey, remember that less is more. The longer it is, the less likely it is that people will take the time to do it. People get bored with long surveys, and usually won't even bother to look at a survey that is more than a page and a half long. Also, requiring long answers may lose your audience. Through editing and condensing, you should try to keep your survey down to one page.
What it is you want to know and the method of survey (e.g., phone survey, mailed survey) will also influence the length of your survey. Phone surveys, for example, can take a little longer to complete.
Once you've decided on your method, you can go on to write your questions. We'll talk in more detail about distributing your survey later on.
Example: 1997 Youth Risk Behavioral Survey sampling
The 1997 YRBS used a type of sampling called cluster sampling. In cluster sampling, the entire population is divided into groups, or clusters, and a random sample of these clusters are selected. For example, age group or geographical location determined the YRBS's clusters. All observations in the selected clusters are included in the sample. This technique is used in large-scale surveys where it may be more convenient to sample clusters than to do a pure random sample.
Write your questions.
When preparing the questions, bear in mind that they can take many forms. Questions might be:
For example: "Why is it important to use condoms?" These questions are used when you want to find out what leads people to specific behaviors, what their attitudes are towards different things, or how much they know about a given topic; they provide good anecdotal evidence. The drawback to using open-ended questions is that it's hard to compile their results.
For example: "Do you use condoms?" These are used when the information you need is fairly clear-cut, i.e., if you need to know whether people use a particular service or have ever heard of a specific local resource.
For example: "When I have sex, I use condoms... a) every time, b) most times, c) sometimes, d) rarely, e) never." These allow you to find out more detailed information than closed-ended questions, and the results can be compiled more easily than open-ended questions.
1 = strongly disagree
2 = disagree
3 = undecided
4 = agree
5 = strongly agree
If you want to weed out neutral and undecided responses you can use an even-numbered scale with no middle "neutral" or "undecided" choice. In this situation, the respondent is forced to decide whether he or she leans more towards the "agree" or "disagree" end of the scale for each item. The final score for the respondent on the scale might be the sum of his or her ratings for all of the items.
Example: Using the Likert scale
Here are a few sample survey questions in Likert scale format, done without a neutral category:
Please check the answer indicating your reaction to the questions listed below.
The questions you ask depend on the audience you're trying to reach and the information you're trying to obtain.
For example, for demographic information (e.g., questions that determine where people are from, their ages, and their incomes), you should make the survey all check-offs, yes/no questions, and fill-in-the-blank questions so that it?s as easy as possible to complete.
Creating surveys people will answer with the Total Design Method
Low response rates are a major problem with surveys; it's common for the response rate to be as low as 30%. One way of avoiding low response rate is to use the Total Design Method, which was developed by Don Dillman of Washington State University. Dillman's method has been shown to yield an average return rate of 73%.
Questions should be worded carefully in order to yield exactly the information you're looking for.
To make sure your survey works the way you want it to, try it out on a few members of the population you're aiming at before you actually distribute it.
Guidelines for writing your survey questions:
Example: 1999 Youth Risk Behavioral Survey questions
Here are a few examples of questions from the 1999 YRBS.
A. 0 times
B. 1 time
C. 2 or 3 times
D. 4 or 5 times
E. 6 or more times
A. Yes
B. No
A. I have never smoked a whole cigarette
B. 8 years old or younger
C. 9 or 10 years old
D. 11 or 12 years old
E. 13 or 14 years old
F. 15 or 16 years old
G. 17 years old or older
A. 0 times
B. 1 or 2 times
C. 3 to 9 times
D. 10 to 19 times
E. 20 to 39 times
F. 40 or more times
A. I have never had sexual intercourse
B. 1 person
C. 2 people
D. 3 people
E. 4 people
F. 5 people
G. 6 or more people
How do you distribute your survey?
There are several strategies for distributing surveys. We'll talk about the most common one -- direct mail -- in the most detail, but there are many methods to choose from and there is no one perfect method. You may want to use a combination of methods.
Here are a few thoughts to help you decide on your method:
For example, if you are doing a survey on sexual risk behaviors, people may be uneasy telling an interviewer how many partners they've had or other such details.
Direct mail
Direct mailing your survey to people whose addresses are known is the most common strategy. Distributing a survey by mail has a high percentage of non-responders (you're lucky if 30% respond, although it tends to be higher in small communities), but it's a lot easier than many other methods and takes less staff hours.
Interviews and phone surveys
For those who have difficulty reading or using printed materials, or for surveys that require more in-depth answers, interviews might be the most appropriate thing for you to do. Phone surveys work similarly to face-to-face interviews, so we've grouped these two methods together.
Drop boxes
Agencies that have relatively frequent contact with clients -- such as once a month -- you may find that setting up a drop box in their offices are a good source point for distributing surveys. This may also be a good option for agencies that have an incomplete mailing list. It can also be a good way to contact clients of other agencies who have little contact with your group or agency. However, if you use this method of distributing surveys, consider using it along with at least one other method of distribution, because only those already using the services can respond.
Media distribution
For general distribution, publishing a survey in the local paper or attaching a survey to your newsletter might be a good idea.
Convenience sampling
Taking surveys in a public place -- setting up a booth or table in the parking lot at a local discount store, on the sidewalk in the shopping district, etc. -- provides an opportunity to get some exposure for your organization.
Group administration
If your group or organization tends to have large group gatherings, providing surveys to everyone who attends a particular gathering might be a really efficient way for you to gather information. Examples of gatherings where you might want to distribute your survey would include: immunization clinics, commodity food distribution sites, health fairs, and meal sites for older adults. If you want to give your survey out at some sort of group meeting or gathering, get the group's director to put you on the agenda. At the meeting, introduce yourself and explain the purpose of the survey. Then distribute the survey, answer any questions, and collect completed surveys. Don't forget to thank everyone for their participation!
Using multiple methods of distribution
You can combine or adapt two or more of the above methods to suit your own purposes, if you'd like. If more than one method is used, each survey should include instructions that each citizen should complete only one survey. So, for example, if you're having people complete surveys at a booth at the county fair, they should not complete the survey if they've already completed one that came in the mail to their homes.
Collecting the surveys
Soon after the surveys are distributed, some of them will begin to arrive at the sponsoring organization. Here are the steps you should take to collect your surveys:
Example: Administering and collecting the 1997 Youth Risk Behavioral Survey
The CDC wanted to do everything it could to protect the students' privacy and insure that questions would be answered honestly while completing the YRBS. In order for the survey to be administered voluntarily and anonymously, it was done in a self -administered written questionnaire containing 84 multiple-choice questions. Before the surveys were administered, parental permission was obtained through whatever methods those local schools used. Students recorded their responses to the questionnaires on computer-scannable answer sheets, further allowing for anonymity.
How do you analyze and compile the results of your survey?
Now that you've gathered the completed surveys, you'll need to figure out the results. Sometimes all you have to do is tabulate the results -- that is, add them up and display in a table.
For instance, if 100 questionnaires were returned in a survey about problems in the neighborhood, you just need to count the answers. Let's say that there was a question asking what people felt was the biggest challenge facing the neighborhood; 70 people mentioned law enforcement, 10 cited transportation, 15 marked potholes, and 5 said noise. The result in cases like this is clear.
However, analysis can be far more complicated than that. If you're looking,
for instance, at how people feel about a service or problem, you may end up with a lot of answers to open-ended questions that are apparently unrelated. In this case, you will need to try to find patterns.
Once you've done that, what do these numbers mean? Well, you will need to look at the overall survey to see how each percentage compares to the others.
For example, what questions had the highest proportions of similar responses?
We suggest that you write up a brief report -- one page is sufficient -- summarizing the results of the survey. In your report, look for any patterns -- do people in a particular part of town feel more strongly about a particular issue than those in other areas?
Share this information with your staff. Get their feedback and discuss whether any further surveying needs to be done before completing.
Now that you've figured out what the results mean, you need to decide what to do with them. To whom are you going to communicate them, and how? In case of a community initiative, the results should be made public as soon as possible so that members in the community and community leaders can be made aware of a problem or potential problem and start working to solve it. If other similar surveys have done in the same area, you may want to compare your results with the other surveys' results.
An organization conducting a survey about its' services might want to use results to provide a better service or to change a current policy to a more efficient one. In a situation where funding is at stake, the results would need to go to the funder to convince the funder of the need for new or continued support. The results could also be used by the organization itself to determine where and what kinds of services are needed. The 2003 Youth Risk Behavior Survey results will give you an idea of what a survey result report looks like.
A well-executed survey can provide your initiative with a wealth of information about your constituents and their needs. We hope this section has given you the tools you need to conduct surveys that are effective and that give you the information you need to serve your constituents better!
Contributor
Chris Hampton
Marcelo Vilela
Online Resources
Kalsbeek, B. (1995). How to Collect Survey Data.
Kalsbeek, B. (1995). How to Plan a Survey.
Hassett, Nancy. (1997). Sample design issues. In FOCUS: An information newsletter for health care professionals, Winner 1997.
de Leeuw, J. (1997). Sample Size Calculator.
Niles, R. (1999). Statistics Every Writer Should Know.
Easton, V.J., and McColl, J.H. (1999). Statistics Glossary, v.1.1.
Print Resources
Bailey, K. (1978). Methods of social research. New York, NY: Free Press.
Berg, B. (1995). Qualitative research methods for the social sciences. Boston: Allyn and Bacon.
Fink, A. (1985). How to conduct surveys: A step-by-step guide. Beverly Hills: Sage Publications.
Community Toolbox
KU Center for Community Health and Development. (2023). Chapter 3, Section 1: Developing a Plan for Assessing Local Needs and Resources. Lawrence, KS: University of Kansas. Retrieved January 4, 2023, from the Community Tool Box: https://ctb.ku.edu/en/table-of-contents/assessment/assessing-community-needs-and-resources/develop-a-plan/main