1 of 31

TFCA DATA COLLECTION

Today we are talking about some key concepts related to TFCA data collection. My name is Sara Cooper. I am the Director of CEPEG, the Community Epidemiology and Program Evaluation Group at the CU Cancer Center and the Colorado School of Public Health. I am so pleased to be co-presenting with my colleague, Dan Simon, the STEPP Epidemiologist from the Center for Health and Environmental Data at CDPHE. I also have a few of my Senior Evaluators on Zoom with me to help monitor the chat in case you have questions and don't feel comfortable shouting them out at us during the presentation.

Yesterday, my team showed off a very cool new survey toy called Poll Everyone and I'm a little grumpy that 18 hours was not enough time for me to find it, learn how to use it, and get it on to my slide deck with any confidence, so we're going to do one last ice breaker the old fashioned way- use a zoom reaction to show how confident you are about finding, analyzing and interpreting data.

2 of 31

AGENDA

Introductions
Digging into the Data: Where do we start?
Right-Sizing Your TFCA Assessment
Quantitative Analysis
Qualitative Inquiry
Resources
Questions

3 of 31

Meeting Agreements

Be curious, ask questions
Value multiple perspectives
Challenge ideas not people
Assume positive intent
Offer grace for us and for yourself

AS we dive in to this I'd like to run through some meeting agreements so we're all on the same page- Right now is the time to be curious and ask questions. Chances are if something is unclear to you, someone else on the zoom doesn't understand either. This is a safe place to take risks and make mistakes. I learn something new every time I present. It's one of the reasons I enjoy doing it so much. Let's come together from a place of empathy- we're all seeking to understand new skills so Let's challenge ideas not people. And while I believe wholeheartedly that we should accept conflict as a catalyst for learning, I also hope that we are all striving for the common good together and we should assume positive intent. Finally, I know I'm not perfect. My kids tell me every day how often I make mistakes. We're all busy. Working in Public health is challenging. I know we're all doing the best we can. So I'll ask you for some grace, and ask you to make sure you save some grace for yourself as well.

4 of 31

Community Epidemiology and Program Evaluation Group (CEPEG) at CU Anschutz

CEPEG is a diverse group of multi-disciplinary researchers committed to advancing the health and wellbeing of all Coloradans through innovative research, intervention, and evaluation.

CEPEG Evaluators…

Share research and evaluation trainings and resources
Co-design data tools with community partners
Offer individualized feedback on evaluation plans, project theory diagrams, and project flow diagrams
Respond to evaluation questions at any time
Assist grantees in thinking through program challenges and revising evaluation plans because the work inevitably changes!

5 of 31

Dan Simon, PhD �STEPP Epidemiologist�Center for Health & Environmental Data, CDPHE�daniel.simon@state.co.us

I can help you…

Find secondary (i.e., surveillance, quantitative) data for your community

Use the TFCA secondary data tool to get a snapshot of tobacco and nicotine use in your community

Interpret the available data and identify areas of strength and opportunities for growth in your community

Complete the TFCA Commercial Tobacco Behavior Data Review Sheet

6 of 31

Data inform your Tobacco-Focused Community Assessment (TFCA)… Where do you start?

Use your End of Grant Report and your knowledge of your community to answer the following questions:

At the end of FY24-26…

What was accomplished/ what was finished?
What was still in progress and should continue?
What new needs or issues are emerging in your communities?

Answers to these questions help you focus your FY27-29 TFCA data analysis.

As part of the TFCA process, you have access to a wide range of data on Colorado commercial tobacco/nicotine use. Do you look to local data? Do you start with regional or statewide data? What questions do you ask? I know it can feel overwhelming trying to figure out where to start. It is easy to get caught in a fishing expedition or lost down a rabbit hole that isn't particularly informative or productive. As we were thinking about the best way to help you all narrow down your initial formative questions, we realized that you already had the tools you needed. At the end of the FY24-26 grant cycle, STEPP asked you to reflect on your work throughout the three-year period. In your End of Grant Report, you were asked to consider three questions:

· what had been accomplished

· what was still in progress and should continue

· what new needs/ issues were emerging in your communities.

Your responses to these prompts are the jumping off point to create a series of analytic questions that will guide your data analysis. Right-size your assessment by using what you already know…

7 of 31

Right-Size Your Assessment… (examples)

For completed goals that may require further effort/ exploration, goals still in progress and emergent areas of concern:
TFCA Goal EXAMPLE	Need for Future Effort/ Exploration	Priority Population	Geographic Area(s) Served	Other Considerations	Analytic Question What are…	Source
Strategy 1.2: Integrate Tobacco Prevention and Cessation into School Env.	YES	14-18 yrs (HS)	XX County	Vape use; other nicotine product use (ONPs); flavor product use; co-use (incl marijuana/ cannabis)	Trends in nicotine use among High School (HS) populations in XX county since 2024	HKCS
	YES	14-18 yrs (HS)	XX County		Trends in nicotine/ tobacco/ cannabis co-use (poly-use) in HS and in XX county since 2024	HKCS
	YES	14-18 yrs (HS)	XX County		Trends in flavor nicotine product use by product type among HS in XX county since 2024	HKCS
	YES	14-18 yrs (HS)	XX County	Primary Data	Effective school district policy initiatives/ challenges to effective district policy communication/ enforcement	Key Informant Interviews
	YES	14-18 yrs (HS)	XX County	Primary Data	Opportunities, topics and effective channels for school district professional development/ TA and training on emergent topics in youth nicotine/ tobacco	Key Informant Interviews

I hope this worksheet looks familiar- it is the Right-size your Assessment worksheet from the Phase 1 Guidance document. The purpose of this worksheet is to help you take the questions from the End of Grant report and turn them into thoughtful analytic questions. When you are ready to start thinking about data, pull out your End of grant report, and the work sheet. Look specifically at your goals and activities that are still in progress and should continue, and new/ emergent areas of concern. Let's take a moment to level-set: It is just fine to have issues that carry over from grant cycle to grant cycle. Some issues are too big to fix in three years. Some issues are too big to fix, period. You just keep searching for solutions, keep fighting, keep chipping away. Sometimes that's the best we can do.

Identify your priority population for each area, the geographic area served, identify any other areas for consideration-

For this example, I've selected Strategy 1.2: Integrating Tobacco Prevention and Cessation into the School Environment. In my County- County X- This is still a serious issue especially in high school so I've specified youth ages 14-18 years as my priority population. I am looking at county-level data and I've identified that I might like to explore vape use, other nicotine products like ONPs, flavored nicotine use, and co-use of products including cannabis. for other topics, I've indicated that these might be better explored through primary data collection.

Then I've specified my analytic question and selected the appropriate data source- if I know what it is.

Each row is a different analytic question or analysis.

Remember- you are also an expert on your community.

8 of 31

Turn your knowledge into analytic questions�Strong questions are SMART

SMART questions are actionable, focus on an outcome, population, location, timeframe, and are relevant to your community(s).

Specific
Measurable
Actionable/ Achievable
Relevant
Time-bound

Instead, use your professional expertise, your organization’s historical experience, and your community knowledge to frame your questions. For example,

"How have youth vaping rates changed among high school students in X County between 20XX and 20YY?“

This Photo by Unknown Author is licensed under CC BY-SA-NC

Avoid overly general questions like, "Are youth vaping rates high?"

9 of 31

You have questions: What’s Next?

Some questions will orient to quantitative data analysis; some orient to qualitative inquiry

Secondary Data
In the TFCA, secondary data are quantitative surveillance-level data
Source	Secondary data are typically collected by someone else
Secondary Data Examples	HKCS BRFSS National, regional or local data

Primary Data
In the TFCA, primary data are qualitative
Source	Primary data are collected and examined by grantees
Primary Data Examples	Key informant interviews Focus groups Community conversations

10 of 31

Quantitative Analysis

TFCA Secondary Data Tool

11 of 31

TFCA Secondary Data Tool

Adult use data

Behavioral Risk Factor Surveillance System (BRFSS): 2022-2024, all adults 18+
Pregnancy Risk Assessment Monitoring System (PRAMS): 2021-2023, women who recently gave birth

Youth use, perceptions, access

Healthy Kids Colorado Survey (HKCS): 2025, high school youth

12 of 31

Additional sources of data beyond the TFCA tool

Regional profiles from the Colorado Health Survey (CHS, formerly TABS) coming next month

Regions 1 & 4 don't have HKCS data – not to worry!

If working in these communities, please reach out to me for alternative youth data

Other (potentially) helpful data tables & dashboards from my team & others at CDPHE

13 of 31

TFCA Data Tool - A Place to Start

14 of 31

Tooltips: Hover over any data point for more info

15 of 31

Helpful links at the bottom of the tool

Link to STEPP-CDPHE website

Download the data directly to your device or learn more about each data source

Higher/Lower than the CO mean estimate

16 of 31

Using the tool for county data?

Example: Park County

17 of 31

HSR 17 vs. Park ...PRAMS & HKCS now available!

18 of 31

Wait...notice any differences between Park & Park's region (HSR 17)?

Orange = Higher than CO average; Blue = Lower than CO average

19 of 31

Goldilocks Scenario

Too little data

What are your first steps?

Just right!

Who would you talk to?

Too much data

What questions might you ask?

20 of 31

Example 1

21 of 31

Better!

22 of 31

Example 2

23 of 31

Qualitative Inquiry

24 of 31

Qualitative Data…

Helps put quantitative data into context
Can help fill gaps in quantitative data
Provides community perspective
Creates opportunities to establish connections
Engages community: collective efforts to identify priorities and possible solutions
Assesses both community readiness and intervention feasibility

Qualitative data collection is the process of gathering descriptive information rather than numbers or numerical data. It allows us to better understand the underlying reasons, opinions, meanings, and motivations behind human behavior. Instead of measuring "how much" or "how many" (which is quantitative data), we focuses on "why" and "how" things happen.

Qualitative inquiry is

Subjective and Contextual: It captures personal perspectives, cultural nuances, and social environments.
Open-Ended: It uses flexible methods that allow participants to explain their thoughts in their own words rather than choosing from a fixed set of answers. For example, you might develop a semi structure interview guide and then follow the discussion where it leads
QI also relies on Small Sample Sizes: It deeply explores a small, focused group of people rather than seeking statistically representative data from a large population.

25 of 31

Collecting Data

In-Depth Interviews: One-on-one conversations designed to explore a participant's personal perspectives, their experiences, their expertise, their background on a specific topic.
Focus Groups: Guided discussions with a small group of people (usually 6–10) to observe how they interact and share ideas about a product, concept, or issue.
Community Conversations: Are structured public discussions that bring together diverse groups who share community perspectives, build consensus on local issues and brainstorm big ideas

Other types of Qualitative data collection include

Observations: Recording behaviors, events, or environments as they occur naturally, often used in ethnographic research.
Open-Ended Surveys: Where you develop a structured questionnaires that allow respondents to type out full-text answers rather- it allows fully open-ended responses rather than forcing responses from multiple-choice boxes.
Document and Artifact Analysis: Examining existing written texts, historical records, emails, or multimedia (like photos and videos) to find cultural or social themes.

26 of 31

Linking Real-World Scenarios to Your Methods�(The Goldilocks Scenario)

No data or data are inconclusive

Environmental or policy scans focus efforts

Community conversations and focus groups identify community priorities, and prioritize strategies and opportunities for action

Data are sufficient to identify critical issues

Key Informant Interviews and focus groups identify opportunities (e.g., identify or strengthen partnerships) as well as challenges (e.g., environmental, financial and political factors)

Too much data: no clear path forward

Community conversations and focus groups highlight community priorities and intervention feasibility

When you look over your surveillance level data, how do you decide what to do next? We’ve identified three scenarios that may seem over-generalized, but are convenient for our purposes. The intention here is to show you why you may consider qualitative data collection. And then how to leverage your surveillance data to inform next steps- developing specific questions about tobacco strategies, and identifying appropriate key informants.

Dan showed you a few examples where there was No data- In these cases, Community Conversations and focus groups are critical to better understand community needs, opportunities for intervention and priorities next steps.

Just because data are sufficient doesn’t mean there aren’t more questions to ask- you can use Qualitative methods to refine strategies/ settings; establish relationships with partners; and get buy-in and feedback from community members and partners so efforts resonate with populations of focus and key groups

Sometimes too much data can have the same effect as too little data. Here, community engagement can help tease out priorities and determine feasibility of potential interventions

27 of 31

Align your purpose with your method(s)

Purpose/ Topic: What do you hope to learn?

Who do you want to talk to? Who knows the most about your topic? How will you find them?

Types of coding techniques include…

Content, Narrative, Thematic, Phenomenological

METHODS

PARTICIPANTS/ EXPERTS

Who is interested in your findings? Who do you want to see them?

AUDIENCE

How will you collect this information? Key Informant Interviews? Focus Groups? Community conversations/ engagement?

ANALYSIS

As researchers and program evaluators, we always ask ourselves the same set of questions when we approach a project. Who, what, why, how and when

These questions allow us to make sure that our methods are aligned with the purpose and goal of our project. They keep us focused.

Observations- will we watch community events unfold and record themes, trends, activities or other events of note as they unfold?

Will we conduct Interviews with experts or with individuals who have specialized knowledge (KII/ Focus Groups)

Will we engage Communities in conversations to identify priorities, build consensus, brainstorm, and problem solve solutions for social change

And are there policy documents or project ( or community) artifacts that we want to review?

When we decide how we plan to analyze our data, will we

Content: looking for patterns in frequencies of phrases or concepts

Narrative: exploring individual stories (e.g., case studies)

Thematic: identifying major themes across datasets

Phenomenological: exploring meaning and lived experiences

28 of 31

Outreach and Recruitment Strategies

One of the hardest parts of qualitative data collection is recruiting participants so I thought we would address this challenge explicitly.

Think back to one of the questions we identified through our right sizing process at the very beginning of this presentation:

We wanted to explore "Effective school district policy initiatives to encourage cessation/ challenges to effective district policy communication/ enforcement" and we thought this topic might be best examined using key informant interviews.

Where do we start- make a list: who would know something about this? Well we already came up with a few ideas: since we are talking about district level policies, we might start there with individuals who make policy at the district level. Step one here is to leverage your existing network- who do YOU know? Who have you worked with? Who do you have a relationship with? Who do they know? Can you tailor your messaging and your communications channels to reach certain groups?

There are different outreach strategies for different types of questions-

Another thing to think about before we move on- important element of outreach and recruitment in community work is reciprocity –making sure to share back with communities and groups who have trusted you with their data. How do you do that in a timely and respectful way that honors their contribution.

engaging with existing groups/existing community priority work.

29 of 31

Going the Distance

Am I confident that:

The people I spoke with represent diverse community perspectives, opinions and cultural dynamics and include those who are most affected by the tobacco problem
I removed or reduced barriers to participating in my data collection efforts
I asked questions that helped me gain a better understanding of tobacco-related issues in my community (population/ health system)
Participants were knowledgeable about tobacco-related issues in the context of the larger community (population/ health system)
Participants had opportunities to share their ideas
All of my questions are answered; no new information is being shared
I can use the qualitative data I collected to inform strategy and setting selection

30 of 31

Meet CEPEG Evaluators and Data Systems Team Members at Public Health in the Rockies

Turning Community Voice and Local Data into Public Health Action

Skills Building Session – Tuesday 8:30 am

Fundamentals of Survey Design

Skills Building Session – Tuesday 8:30 am

31 of 31

If you get stuck…

Watch CEPEG’s webinar on Qualitative data collection and analysis
Check out https://sites.google.com/view/cepeg-resources/evaluation-start-to-finish
Review additional resources at: bit.ly/CEPEGhome
Reach out to CEPEG Evaluators at: EvaluationTA@ucdenver.edu
For secondary data help, contact: daniel.simon@state.co.us