1 of 21

Establishing Clinical Significance Benchmarks and Minimally Important Difference for 17 Industry Standard Measures

Caroline G. Vincent, Eric A. Youngstrom, PhD, Joshua A. Langfus, MA, Jennifer K. Youngstrom, PhD, & Robert L. Findling, MD, MBA

​

Caroline Vincent

2 of 21

Objectives

  • Demonstrate the benefits of quantitative progress assessment
  • Understand the Jacobson and Truax method of reliable change
  • Get connected to helpful EBA tools on Wikiversity
  • Test your knowledge using a clinical vignette

​

2

3 of 21

Background

  • Tracking and reporting patient progress improves service engagement and outcomes (Lambert et al., 2005; Morris et al., 2021)
  • We can use self-report assessments that clients are already taking
  • Assessing significant change aids clinical judgement in understanding if interventions are working (Miller et al., 2015)
  • Change indexes that are measure-specific are initially more labor intensive than clinical judgment (Youngstrom et al., 2015)

3

4 of 21

Nomothetic vs. Idiographic

Nomothetic: the tendency to generalize

    • comparing clients to established norms

�Idiographic: the tendency to specify

    • comparing clients to prior performance

​

In outcome assessment, both are useful (Ashworth et al., 2019)

4

5 of 21

Common ways to measure significant change

  • Clinical practice: Clinical judgement after retest and feedback from client
  • Research: average points of change for the group
  • Pharmaceuticals: 33% reduction in symptoms

5

6 of 21

Why use quantitative measures of change?

They account for:

    • Regression to the mean
    • The standard error of the measure
    • Can contextualize change to see if it is significant

6

7 of 21

Minimally Important Difference (MID)

The minimally important difference (MID) is the change in score of a patient-reported outcome (either beneficial or harmful) that is important and would warrant a change in the patient’s management

    • Common in medical settings
    • Criticized for being too liberal

7

8 of 21

Jacobson & Truax: Reliable Change Index (RCI)

Divide observed change between two time points by the “standard error of the difference” for the measure (z-score)

RCI > 1.65 = 90% reliable �RCI > 1.96 = 95% reliable

...both “two-tailed”

8

9 of 21

Jacobson and Truax: Nomothetic Benchmarks

If the RCI is met, the next step is to pass a benchmark.

    • Away from the clinical group
    • Back into the non-clinical group
    • Closer or crossing over to the non-clinical group

9

10 of 21

10

11 of 21

Example

11

12 of 21

12

13 of 21

13

14 of 21

14

https://bit.ly/hgapscs

15 of 21

15

​

Cut Scores

Critical Change Score for RC

MID

d ~.5

A

B

C

95%

90%

SE difference

CBCL Total

Male 6-11

3.8

57.1

39

11.1

9.3

5.7

12.8

Male 12-17

1.7

63

40

11.6

9.8

5.9

12.8

Female 6-11

-

55.1

35

10.7

9

5.5

12.6

Female 12-17

-

58.2

37

11.6

9.7

5.9

13.1

TFR Total

Male 6-11

-

70.1

38

13

11

6.6

13.9

Male 12-17

-

75.4

40

14

11.8

7.2

14.8

Female 6-11

-

55.1

27

11.7

9.9

6

12.9

Female 12-17

-

54.6

29

12

10.1

6.1

13.3

YSR Total

Male 11-18

-

113

54

18.3

15.4

9.3

14.8

Female 11-18

1

117

59

18

15.2

9.2

14.6

Beck Depression Inventory Mixed Depression

4

22

15

9

8

4.8

3.4

16 of 21

16

For more detail go to bit.ly/hgapsjt

17 of 21

Clinical Vignette: Leah

Lea is an 18 year old white female in high school. She self-referred to the clinic because she has been having a lot of trouble with attention and focusing on course material. Her grades are dropping rapidly, and she is getting anxious and worried about graduating (and IF she will graduate). As her anxiety spirals up, her grades are coming down.

17

For more detail go to bit.ly/HGAPSLea

18 of 21

Clinical Vignette: Assessment

After intake, Leah identified wanting to work on symptoms of depression. Her clinician asked her to take the General Behavior Inventory: 7Up 7Down online in the HGAPS Assessment Center

​

Her Results

Down Score: 13

Up Score: 2

After Interpersonal Psychotherapy

Down Score: 8

Up: 1

18

bit.ly/HGAPSAC

19 of 21

Was Leah’s improvement significant?

19

Down Score: 13 to 8 Up Score: 2 to 1

( 1 ) RCI ( 2 ) Passing a benchmark

20 of 21

Takeaways

  • Quantitative progress assessment can be helpful to track patient progress
  • The Jacobson and Truax model incorporates both idiographic and nomothetic measures of change
  • You now have more tools in your clinical toolbox of assessment

​

20

21 of 21

Thank You!

Contact: carvin@live.unc.edu

For more resources www.hgaps.org

OSF: https://bit.ly/hgapscs

21