1 of 18

Breakout 4C�Measuring Success: Program + Outcomes Evaluation for Care Farming

Jackie Farrell, OTD

Occupational Therapist - Vinfen

Adjunct Faculty at Boston University

Sargent College of Health and Rehabilitation Sciences

2 of 18

Why is Program Evaluation Important?

“There is a general trend of evidence to suggest that care farming may be effective for some service user groups; however, this evidence is limited. Small study sizes, evaluations involving mixed service user groups, the use of multiple and sometimes unvalidated outcome measures, short follow-up periods and absence of missing key outcomes that fit with theory have all hampered the development of a more robust evidence base... Despite the current lack of conclusive evidence from RCTs and robust effectiveness studies, the indications from the available evidence suggest that CFs are a promising alternative or adjunct to mainstream approaches for mental ill health. Lack of choice, gender inequalities and overburdened statutory services indicate the need for a credible alternative treatment option. A concerted effort to increase awareness among commissioners of health care, front-line service providers and potential service users about care farming, and how and for whom it may work, is needed.”

- Systematic Review of Care Farming by Elsey et al., 2018

3 of 18

Why Look at Program Outcomes?

  • Determine effectiveness of your care farm for your community
  • Communicate care farm benefits to stakeholders/communities of interest
    • Outreach to potential referral sources
    • Marketing to greater community
  • Request funding for care farm services

​

  • Questions to consider:
    • What outcomes are you interested in learning about? What do you want to know about your care farm?
    • Who are the outcomes for? Who are the key stakeholders/COI? What do they want to know about your care farm?
      • Participants
      • Care farmers
      • Funding sources

4 of 18

Types of Outcomes Data

  • Quantitative Data: numerical data – understand “how many” or ”how much” questions
    • How many people served through your farm per year?
    • How many hours per day do individuals typically work at a care farm?
    • How much do depression scores change over time? E.g. pre/post
      • Information collected through survey/questionnaires using standardized measures

​

  • Qualitative Data: non-numerical (opinions, experiences) – understand “how” and “why” care farming is effective
    • What are the qualities of care farms that make them beneficial?
    • How or why are care farms beneficial?
      • Information collected through interview, focus group, surveys, testimonials

(Marjolein Elings, 2011;Platform Youth Farms, 2010; Wageningen UR, 2011)

5 of 18

Quantitative or Qualitative Data?

(Hassink, 2010)

Qualitative

6 of 18

When to Collect Data

  • Summative Assessment: evaluation at a set benchmark to determine final progress
    • Most common way to track outcomes
    • Pre-post (before-after care farming program)
    • Pre-post with long-term follow-ups (3, 6 9+ months after)
    • Control group?

​

  • Formative Assessment: continuous evaluation process used to monitor progress and make changes in real-time
    • Gather information about what is vs. is not working
    • Get feedback from participants

​

7 of 18

How to Collect Data

  • Survey (paper or online)
  • Interview
  • Self-report measures
  • Proxy-report measures
  • Standardized measures

​

  • Considerations
    • Length of measure – consider cognitive capacity
    • Reading level required – consider education level, English language skills
    • Where/when to complete measures – consider motivation
    • Self awareness/insight level

8 of 18

Types of Data

​

Pros

Cons

Survey

  • Can gather data from large number of people
  • Easy to compile data for statistical analysis
  • Unable to gather further information if needed
  • May miss information

Interview

  • Personalized approach
  • Can gather in-depth information and ask follow-up questions
  • Interviewer may bias results
  • Requires extended time

Self-report

  • Get first-person perspectives of their own thoughts and feelings
  • Perspectives inherently biased

Proxy-report (i.e. caregiver, coworker, clinician)

  • Proxy can provide details if individual cannot due to cognitive impairments
  • Provide insight into caregiver perspective
  • Proxy may be biased in their reporting

Standardized measures

  • Deemed more rigorous for research purposes
  • Allows for comparison across other settings
  • Training may be required to administer the measure
  • Rigid protocols to administer may make them unfeasible at farm

9 of 18

Selecting Outcome Measures

  • Question 🡪 Identify 1) population and 2) outcome 🡪 Find Measure

​

​

Cacciatore et al., 2020; Lee et al., 2019

Does care farming improve communication skills in families with young children?

Population = families (mother/child dyad)

Outcome = communication skills

Outcome Measure = Parent-Children Communication Inventory

Does care farming decrease traumatic grief for adults who experience the loss of a loved one?

Population = adults

​

Outcome = traumatic grief

Outcome Measure = Traumatic Grief Inventory Self-Report

10 of 18

Potential Outcomes for Care Farming

  • Mental health: self-efficacy, self-esteem, stress, coping, mood, mental status, mental functioning, positive affect, rehabilitation and cognitive functioning, empowerment, resilience
  • Social: social functioning/interactions, group cohesion, recidivism, employment, school exclusion, emotional intelligence, communication skills, conflicts with others
  • Physical: functional performance, physical activity, appetite, eating pattern, sleep
  • Behavioral: drug use, alcohol intake, smoking, risky behaviors
  • Quality of life and well-being

(Marjolein Elings, 2011; McDowell, 2006; Platform Youth Farms, 2010; Wageningen UR, 2011)

11 of 18

12 of 18

How Do I Start?

  • No need to reinvent the wheel – duplicate from other existing research studies
    • Find studies through CFN, Google Scholar, research databases
      • “care farming”, “green care”, “horticultural therapy”
    • Methods section will typically name outcome measures used

​

  • Collaborate with others
    • Local university partnerships and research labs
    • Various fields – psychology, physiology, rehabilitation, social work, environmental, biology, etc.

​

  • Free resources

​

13 of 18

How Do I Start?

14 of 18

15 of 18

How Do I Start?

16 of 18

How Do I Start?

17 of 18

References

Cacciatore, J., Gorman, R., & Thieleman, K. (2020). Evaluating care farming as a means to care for those in trauma and grief. Health & place, 62, 102281. https://doi.org/10.1016/j.healthplace.2019.102281

​

Elsey H, Bragg R, Elings M, et al. Impact and cost-effectiveness of care farms on health and well-being of offenders on probation: a pilot study. Southampton (UK): NIHR Journals Library; 2018 Feb. (Public Health Research, No. 6.3.) Chapter 2, The impact of care farms on quality of life among disadvantaged populations: a systematic review. Available from: https://www.ncbi.nlm.nih.gov/books/NBK487292/

​

Lee, A. Y., Kim, S. O., Gim, G. M., Kim, D. S., & Park, S. A. (2019). Care Farming Program for Family Health: A Pilot Study with Mothers and Children. International journal of environmental research and public health, 17(1), 27. https://doi.org/10.3390/ijerph17010027

​

McDowell, I. (2006). Measuring health: A guide to rating scales and questionnaires (3rd ed.). Oxford University Press.

​

Murray, J., Wickramasekera, N., Elings, M., Bragg, R., Brennan, C., Richardson, Z., ... & Elsey, H. (2019). The impact of care farms on quality of life, depression and anxiety among different population groups: A systematic review. Campbell Systematic Reviews, 15(4), e1061.

​

https://www.sralab.org/rehabilitation-measures

​

18 of 18

Let’s stay in touch!�Jackie Farrell, OTD, OTR�jmf13@bu.edu