AI AND ACCREDITATION: DATA ANALYSIS �
BETH KUBITSKEY, DIRECTOR – CAEP
ANNE TAPP-JAKSA, PROFESSOR - SVSU
MARCIA FETTERS, PROFESSOR - WMU
What We Will Achieve Today
Apply Ethical Frameworks: Evaluate the use of AI tools in accreditation processes using the FATE framework.
1
Execute Multi-Modal Data Analysis: Independently conduct qualitative thematic synthesis, quantitative statistical breakdowns, and data visualization on employer survey data.
2
Synthesize Actionable Plans: Translate AI-generated data insights into actionable, program-specific strategies for continuous improvement.
3
OPENING QUESTION: AI WILL FUNDAMENTALLY RESHAPE ACCREDITATION WITHIN 5 YEARS?
Agree
Unsure
Disagree
INTRODUCTION TO AI TOOLS
Tool | Best Use Case | Free Tier "Stamina" | Why for this Survey? |
NotebookLM | The Synthesis Pro | High (100 Notebooks / 50 Sources each) | Best for Qualitative Synthesis. It doesn't just analyze data; it creates a "brain" of your documents. Perfect for turning principal comments into a coherent report. |
Google Gemini | The "Play Around" Choice | 30 prompts / day | Most generous for groups. Use it for the initial data cleaning and creating charts from the Likert scales. |
Microsoft Copilot | The Corporate Standard | 30 turns / session(Web only) | Great for familiarity. Free users must use the web version to upload files, as in-Excel features are now paid. |
Claude (Anthropic) | The Qualitative Expert | ~15 messages / 5 hrs | Unmatched for finding "tone" and nuanced sentiment in employer feedback. |
ChatGPT (OpenAI) | The "One-Shot" Demo | 2 Data Analyses / day | Strong logic, but the strict file limit makes it a "demo-only" tool for free users. |
PRACTICUM ORIENTATION: THE DATA SET (AY 2023)
SPRINT 1: QUALITATIVE SYNTHESIS (12 MINUTES)�TASK 1: THEMATIC ANALYSIS
Goal: Turn unstructured employer feedback into coherent themes
Your Prompt: "Analyze the open-ended comments in this survey. Identify the top 3 positive themes and the top 3 areas for growth across all programs. Provide a quote to support each theme."
https://tinyurl.com/3bny4j4c
SPRINT 2: QUANTITATIVE BREAKDOWN & ACTION (13 MIN)�TASK 2: STATISTICS & ACTIONABLE INSIGHTS
Your Prompt: "Complete a statistical analysis of the Likert questions. Disaggregate the data by program (Column 2). Calculate the mean, median, mode, standard deviation, and percent 3 or 4 scores for each question., identify the top 3 actionable steps to improve candidate preparation for each program"
https://tinyurl.com/3bny4j4c
SPRINT 3: MIXED METHODS
1. "Analyze the attached principal satisfaction survey data. Complete a statistical analysis of each Likert question, including the percentage of 3 or 4 scores. Compare across programs (column 2). Are there statistical differences amongst them? Do a sentiment analysis of the open-ended questions. Disaggregate by programs and determine if there are any differences. Do a thematic analysis of the open-ended questions and identify the top 3 positive comments and top 3 negative comments. Compare Likert questions with open-ended questions to determine whether there are relationships. Identify the top 3 action items for each program based on these data."
2. “Create a report describing the findings by program and overall”
TABLE DISCUSSION
WHERE CAN AI IMPROVE DATA ANALYSIS AND REPORTING AT YOUR OWN INSTITUTIONS?
SYNOPSIS
AI is a Co-Pilot, not an Autopilot: AI supports data organization, evidence alignment, and reporting, reducing faculty workload.
Lead with FATE: Always evaluate AI tools for Fairness, Accountability, Transparency, and Ethics to mitigate bias.
Drive Equity with Data: Use predictive analytics and AI dashboards to flag at-risk candidates earlier and bring visibility to performance differences across subgroups.
Embrace Agility: Transition toward agile accreditation models featuring modular evidence systems and real-time feedback loops to keep pace with innovation
CONCLUDING DISCUSSION
(trust but verify)
https://tinyurl.com/464ubuh7