1 of 18

Using Latent Profile Analysis to Identify Performance Patterns and Improve Licensure Outcomes

Benjamin Shultz, PhD

Director of Assessment; Research Assistant Professor

​

Kristen L Goliak, PharmD

Associate Dean for Academic Affairs and Assessment

Clinical Associate Professor

2 of 18

Learning Outcomes

Explore how structured assessments can serve as early indicators of success on high-stakes exams.

Consider the role of structured preparation versus self-directed study in high-stakes exam readiness.

3 of 18

Background

North American Pharmacist Licensure Examination (NAPLEX)

4 of 18

Theoretical Foundation

    • Exam structure and passing standard revisions
    • Pre-pharmacy coursework and preparedness
    • Lack of curricular effectiveness

Why the decline?

    • Third-party prep resources
    • Targeted intervention
    • Mock exams

How to respond?

5 of 18

Research Questions

What factors contribute to success on the licensure exam among our students?

Can we identify students who are likely to struggle early enough to design targeted interventions that improve their chances of success?

6 of 18

Methodology

Structured Preparation Curriculum

License provided to all 4th year students

7 mandatory assessments during year

Tracked usage and performance

Linked to Other Metrics

NAPLEX pass/fail result

Didactic performance in core content

Days between graduation and exam

7 of 18

Methodology

Latent Profile Analysis (LPA)

Identify hidden (latent) groups

Determine the number of profiles

Interpret profile characteristics

Comparison of Groups

Check for statistical differences

Determine importance of variables

Compare licensure pass

8 of 18

Methodology

Population

    • 119 students (Class of 2024)
    • Identified NAPLEX results

Variables

    • Days to exam
    • Prep exam average
    • Prep questions attempted
    • Minutes reviewing
    • Core content exam average

9 of 18

Analysis

Low Performers

35 students (29%)

Moderate Performers

63 students (57%)

High Performers

21 Students (18%)

10 of 18

Analysis

11 of 18

Analysis

12 of 18

Analysis

13 of 18

Analysis

14 of 18

Analysis

Significant differences across profiles (p<.001)

Exam Timing, Prep Performance, Didactic Performance

No statistical differences across profiles (p>.05)

Questions Attempted, Minutes Reviewed

15 of 18

Argumentation

Explore how structured assessments can serve as early indicators of success on high-stakes exams.

Consider the role of structured preparation versus self-directed study in high-stakes exam readiness.

16 of 18

Conclusions: Key Take Aways

Adaptive Preparation Strategies

Formal Checkpoints During Curriculum

Early Intervention is Key

17 of 18

Potential Impact

    • Capstone Projects
    • Program Readiness

Flexible Methodology for Other Milestones

    • Curricular Alignment with External Targets
    • Optimal Use of Third-Party Resources

Curricular Effectiveness

18 of 18

Questions

Contact Information

Benjamin Shultz: bshultz@uic.edu

Kristen Goliak: kgoliak@uic.edu

​