1 of 26

While we get started…

Visit localhousingsolutions.org
Click on “housing data” in the menu to explore the Housing Needs Assessment tool and other resources

Welcome!

2 of 26

Welcome to Data Talks at the Housing Solutions Lab!

Introduction to surveying residents about housing needs

Fireside chat with Montana James

Q&A

Agenda

3 of 26

4 of 26

Data Talks 4 | Understanding Housing Quality

Code enforcement data, property inventories, and more

June 2024

Data Talks 5 | Presenting Data to the Public

Storytelling with local data

August 2024

Upcoming Data Talks

in Summer 2024

5 of 26

Why conduct a resident survey?

To understand resident needs and preferences and inform a local housing strategy
To better understand a specific problem, or improve a specific program
Sometimes, it’s required (e.g. as part of an Assessment of Fair Housing or community needs assessment to apply for HUD funding)

6 of 26

Surveys can provide new insights

Reach a more diverse group than attends public meetings
Collect information that is not available through administrative or other data
Build communication channels with residents and stakeholders
Motivate and inform policy change

7 of 26

Surveys also come with challenges

Surveys can be expensive
They are prone to sampling bias
Surveys can be difficult to design so that they yield the information you want
Surveys set an expectation from the community that results will be communicated and action will be taken

a. May include cost of providing incentives; staff time or contracting with a firm for survey design, administration, and/or analysis; survey technology or envelopes/postage and other materials; cost to promote the survey via different communication channels, etc.)

b. Some people are more likely to see and respond to a survey than others, so it may reach a sample that is not representative of the whole)

c. Surveys can be difficult to design so that they yield the information you want. Order, wording, length matter – people may experience “survey fatigue,” skip questions, or provide answers that are unclear or incomplete.

d. Surveys can set an expectation from the community that results will be communicated and action will be taken. They can be an opportunity to deepen engagement, but also daunting to figure out how to do well.

8 of 26

Survey types and goals

Program Evaluations

Attitudes

Needs

Typically cross-sectional

May be longitudinal

Just as they may have a variety of reasons for doing a survey, cities may have a variety of survey goals, for example:

a. Needs assessments: Understanding resident satisfaction, wellbeing, housing conditions like quality, crowding, cost, conditions, unmet needs. These can also be targeted to cover a specific topic or designed to reach a particular group

For example, Kansas City regularly conducts resident surveys to understand housing and other needs and assess progress towards meeting them.

b. Attitudes and perspectives: Surveys to gather community input or priorities, perceptions of different housing policies or directions for a local housing strategy

For example, the Missoula survey we’ll hear about during the fireside chat gathered feedback on specific strategies proposed as part of A Place to Call Home

c. For Program evaluation purposes, to understand a particular intervention or program

For example, the city of Oakland surveyed residents who applied for emergency rental assistance to understand their need for assistance, their experience with the application process, and the impact of receiving assistance for those who did

Surveys can also be either cross-sectional (most common) or longitudinal (following a population, cohort, or panel over time – incredibly time-consuming and therefore expensive; high attrition rate)

9 of 26

Survey methods

10 of 26

Survey methods

On-line
Phone
Text
Mail
In-person
Paper

Sometimes in combination to address non-response or reach diverse respondents.

11 of 26

Online surveys

PROS	CONS
Inexpensive Potential for a large radius Quick response time Easy to compile data Skip logic can help customize the survey and keep it short	Low response rates Incomplete responses Not everyone can be reached online, and particularly via email, due to the digital divide Not everyone is comfortable with online platform

12 of 26

In-Person Surveys

PROS	CONS
More personal engagement Gathers nuanced responses Allows for clarification Typically high response rate Potential to reach groups who do not otherwise engage with local government	Extremely labor- and time-intensive both to administer and compile Survey administrators may require training to remain safe and avoid bias

13 of 26

Best

practices

14 of 26

Enlist partners

Partner with other stakeholders to conduct the survey
Tap educational institutions
Enlist needed expertise

15 of 26

Designing the survey

Make sure you have concrete research questions and goals
Survey items should reflect those goals
Make the survey accessible:

Keep it short
Keep it simple
Translate

Borrow validated survey items that have been tested
Beta-test and refine

Make sure you have a concrete set of research questions and a clear idea of what you are looking for, as your compass for design.
Include questions that speak to those research priorities, and try to avoid survey mission creep to ask things that are nice to know but not really need to know.

A longer survey is not a better survey

If possible, borrow survey items that have been tested and effective in other communities, and speak to your specific research goals. There are validated measures of housing stability and housing cost, that have been tested in other contexts.
Make the survey accessible
Keep individual questions and overall survey short; Longer surveys tend to have lower response and completion rates
Avoid too many open-ended questions, which can be hard to interpret and code
End with demographic questions if possible, instead of leading with them;
if the survey addresses a sensitive topic, look for ways to minimize potential trauma and discomfort
Pay attention to phrasing, vocabulary, question and response order – write simply and clearly
Translate the survey into appropriate languages, and have translations checked to make sure they are correct and culturally appropriate
Beta-test your survey with a small, representative group and collect their feedback
Consider the survey format and if appropriate, use tools like progress bars to motivate respondents
Use a tool or service to review your survey’s difficulty level.

16 of 26

Administering the survey

Create a clear sampling and recruitment strategy

Survey firms, local nonprofits, and universities can help determine how to achieve a representative sample

Work hard to get the needed response

Multiple reminders
Consider incentives
Leverage trusted community members and organizations
Allow time to address slow or low response

Communicate how you plan to use the results

Create a sampling/recruitment strategy to make sure your survey reaches a representative group

Different ways of sampling, including probability-based (random, stratified, systematic, or cluster) and non-probability-based (quota, convenience, purposive, snowball)
You will likely need to enlist help with a sampling strategy. Survey firms can help (Root Policy Research is a firm based in Denver, CO that Stephen Whitlow recommends), but so can local nonprofits (Kalamazoo example – the city worked with the nonprofit Upjohn Institute to do the survey) or universities (UPenn example)

Be ready to work hard to get a representative sample, or to reach your sample size goals

Issue multiple reminders
Consider offering incentives (gift cards, raffle)
Leverage existing touch points
Partner with trusted community members and organizations to promote and administer the survey.

Compensate community members who devote their time/expertise

As part of your outreach and recruiting respondents, make sure respondents know you plan to use the results

17 of 26

Analyzing survey responses

Have an analysis plan
Break out results by race, geography, age, and other key subgroups as possible
Understand the level of certainty your survey sample provides
Share the results with residents and stakeholders

18 of 26

Examples

19 of 26

Surveying residents and in-commuters

Boulder, CO

LINK

In early 2014, the City of Boulder conducted an online “Housing Choice” survey to understand housing preferences and barriers to living in Boulder
The City partnered with major employers and the university to target the survey to in-commuters and students
The survey captured:

1,643 residents
1,405 in-commuters
457 students

The City hired a firm, BBC Research, to lead analysis

20 of 26

Using polls to motivate change

Bellevue, WA

LINK

A coalition of advocacy groups partnered with a professional pollster (Change Research) to survey residents about housing affordability in Bellevue in 2022
Questions were framed to motivate policymakers
The group reached residents through text messages and targeted ads on social media
The survey captured 475 residents and used demographic weighting to adjust for bias

21 of 26

Informing a housing strategy

Kalamazoo County, MI

LINK

Kalamazoo County worked with the W. E. Upjohn Institute, a local research nonprofit, to conduct surveys with housed and unhoused residents
The county sent mailers with an online survey link to 35,500 randomly selected addresses, and promoted the survey on social media
Homeless service providers volunteered to administer the survey to unhoused clients through computers, phones, and tablets
The survey results informed the County’s housing needs assessment and strategy process

22 of 26

housingsolutionslab.org

Martha Galvez, Executive Director

martha.galvez@nyu.edu

Thank you!

23 of 26

24 of 26

Phone Surveys

PROS	CONS
Potential for a large radius Relatively inexpensive	May be difficult to assemble phone numbers May be seen as intrusive

25 of 26

Mail Surveys

PROS	CONS
Potential for a large radius Useful for geographic targeting	Expensive (postage costs) Less likely to reach residents with informal or insecure housing tenure Time-intensive and slow High non-response rate

26 of 26

An in-person survey to maximize engagement

Missoula, MT

LINK

In March 2019, the City of Missoula worked with graduate students at the University of Montana to design and administer an in-person survey
The survey was designed to gather residents’ feedback on a specific set of housing policy recommendations
Students canvassed 5 neighborhoods that had been underrepresented in the planning process, reaching 136 Missoulians for one-on-one surveys