Preparing (and Collecting Data from) Self-Reliant Teacher Education Program Completers
Robert Bruce Scott, Ed.D.
1
A Rallying Cry from the Year 1841
And truly it demands something godlike in him/her who has cast off the common motives of humanity, and has ventured to trust himself/herself for a taskmaster. High be his/her/their heart, faithful their will, clear his/her sight, that he/she/they may in good earnest be doctrine, society, law, to himself/herself, that a simple purpose may be as strong as iron necessity is to others!
- Ralph Waldo Emerson
Self-Reliance
2
CAEP Standard R4.1a Completer Effectiveness
The provider demonstrates that program completers effectively contribute to P-12 student-learning growth.
- Council for the Accreditation
of Educator Preparation
2022 CAEP Standards - Initial Programs
SELF-STUDY REPORT
The self-study report is a collection of evidence and supporting narrative forming the basis for accreditation review, and the first source of information for the site team.
Antithesis-thesis Statement
Although the fundamental concept driving the CAEP accreditation process is self-study on the parts of teacher educators and EPP program administrators, there remains a disconnect between those data-mining exercises and the lack of such opportunities for self-study data-analysis afforded to candidates which would enable program completers to build self-assessment skills for greater autonomy, self-reliance, and resilience to continue growing and adapting to handle unforeseen settings and situations.
Typical Data Answer
EPP utilizes focus groups to collect evidence for R4.1a.
Description of Data Collection:
EPP will train completer participants on measuring student-learning growth, using paired samples t-test.
www.jmp.com/en_us/statistics-knowledge-portal/t-test/paired-t-test.html
Another idea
Let’s reorganize the teacher preparation curriculum to incorporate self-assessment skill-building within key courses as a candidate spirals upwards through a four-year undergraduate program or two or three-year advanced professional programs.
Let’s make self-assessment, self-reliance, and resilience major program objectives towards which candidate improvements are measured from baseline data to key benchmarks that must be met along with growth of knowledge, awareness, and skills in other aspects of the teaching profession.
KEPPS - Kansas Educator Preparation Program Standards for Professional Education
Standard 1: Learner Development
8
KEPPS -
Standard 2: Learning Differences
9
KEPPS -
Standard 3: Learning Environment
The teacher works with others to create learning environments that support individual and collaborative learning, includes teacher and student use of technology, and encourages positive social interaction, active engagement in learning, and self-motivation.
10
KEPPS -
Standard 4: Content Knowledge
11
Standard 5 - Application of Content
5.3.3CK The teacher understands the role information literacy skills play in understanding issues and solving problems independently and collaboratively.
5.3.6PS Through the use of varied technologies and resources, the teacher facilitates the acquisition and application of that knowledge to solve real world problems.
12
Standard 6 - Assessment
Function 1: The teacher understands how to use multiple measures to monitor and assess individual student learning.
Function 2: The teacher understands how to engage learners in self-assessment.
Function 3: The teacher understands how to make informed decisions.
13
KEPPS - Kansas Educator Preparation Program Standards for Professional Education
Standard 7: Planning for Instruction
14
15
KEPPS Standard 8:
Instructional Strategies
Standard 9: Professional Learning and Ethical Practice
Function 2: The teacher uses evidence to continually evaluate his/her practice, particularly the effects of his/her choices and actions on others (learners, families, other professionals, and the community), and adapts practice to meet the needs of each learner.
16
KEPPS - Kansas Educator Preparation Program Standards for Professional Education
Standard 10: Leadership and Collaboration
17
Self-Assessment and Professional Development
3 Articles
“Impact of a Portfolio Program on Self-Assessment Skills Involving General Longitudinal Outcome” (Scartabello, Abate, & Slimak, 2018)
“Self-Reported Medication Adherence and Health Status in Late Adulthood: The Role of Beliefs” (McDonald-Miszczak, Maki, & Gould, 2000)
“Developing Self-Assessment Skills Amongst Doctors in Nepal” (Butterworth, 2010)
Scartabello, Abate, & Slimak, 2018
“Overall, this study found that students’ self-assessment skills related to specific longitudinal outcomes significantly improved through the use of repeated self-assessment entries in a portfolio.”
“This type of portfolio can help improve students’ self-assessment skills while allowing faculty to analyze self-assessment performance, thereby providing another powerful indirect measure of student learning within a program.”
Scartabello, Abate, & Slimak, 2018
Portfolio Self-Assessment Evaluation Rubric (Scartabello, Abate, & Slimak, 2018)
Portfolio Self-Assessment Evaluation Rubric (Scartabello, Abate, & Slimak, 2018)
Suggested enhancements for new program model
23
Self-Assessment and Professional Development
3 Articles
“Impact of a Portfolio Program on Self-Assessment Skills Involving General Longitudinal Outcome” (Scartabello, Abate, & Slimak, 2018)
“Self-Reported Medication Adherence and Health Status in Late Adulthood: The Role of Beliefs” (McDonald-Miszczak, Maki, & Gould, 2000)
“Developing Self-Assessment Skills Amongst Doctors in Nepal” (Butterworth, 2010)
McDonald-Miszczak, Maki, & Gould (2000)
“Self-reports of adherence…are notoriously problematic measures of actual adherence levels because adherence is usually overestimated.”
“It appears…that self-reported medication adherence is influenced by belief-laden constructs that are related to memory performance.”
“[We] expected that beliefs about health care professionals and health locus of control would significantly predict self-reports of adherence.”
McDonald-Miszczak, Maki, & Gould (2000) - Locus of Control
“The Wallston, Wallston, and DeVeils (1978) Multidimensional Health Locus of Control Scale was used.”
“The Internal Locus of Control subscale…assesses the extent to which individuals believe that they are responsible for their own health care.”
“The Powerful Others subscale assesses individual beliefs that physicians, family, or friends are responsible for their health care.”
“The Chance subscale [assesses] individuals’ beliefs that external reasons (e.g., fate or luck) determine health status.”
McDonald-Miszczak, Maki, & Gould (2000) - Results
“The results indicate that neither the medical factors nor the perceptions of health care professionals significantly influenced self-reported adherence.”
“[Only] scores on the Powerful Others Locus of Control scale predicted a significant amount of variance.”
“This result suggests that adults who endorse the important role that powerful others play in their health, also report that they are less adherent to their medication regimen than adults who do not endorse the importance of powerful others.”
McDonald-Miszczak, Maki, & Gould (2000) - Discussion
“[When] older adults are asked to assess their adherence levels, they use their personal belief system rather than conducting retrospective and objective reviews of actual remembering and forgetting episodes.”
“After all, it is a difficult task to accurately remember and review medication taking and forgetting episodes, tally such experiences, and come to some conclusion regarding average adherence for different medications.”
“Thus, a general assessment of adherence may be much like a general metacognitive assessment (i.e., memory self-efficacy) that draws upon beliefs about oneself within the domain rather than actual behaviors.”
Further suggestions for new program model
29
A Rallying Cry from the Year 2019
A teacher’s sense of plausibility is her/his own perception of how learning takes place and how a certain form of teaching can cause or accelerate it. [It] gets modified continuously in the course of teaching which engages the teacher’s mind and classroom decision-making....But it happens only when the teacher has a stake in each lesson, making a decision and taking the chance of success or failure....When teaching is carried out merely or mostly as an act of conformity to stipulated procedures, with no need for the teacher’s ‘on-site’ decisions and no immediate outcome or feedback, it only gets steadily routinized. - N.S. Prabhu
30
Self-Assessment and Professional Development
3 Articles
“Impact of a Portfolio Program on Self-Assessment Skills Involving General Longitudinal Outcome” (Scartabello, Abate, & Slimak, 2018)
“Self-Reported Medication Adherence and Health Status in Late Adulthood: The Role of Beliefs” (McDonald-Miszczak, Maki, & Gould, 2000)
“Developing Self-Assessment Skills Amongst Doctors in Nepal” (Butterworth, 2010)
Butterworth (2010) - the need for self-assessment skills
“One of the key components of life-long learning is the ability to accurately assess ones’ strengths and weaknesses in order to efficiently direct learning to address ‘gaps’ in knowledge or skills.”
“The overwhelming evidence…is that doctors do not self-assess well in all domains of knowledge, skills, and attitudes. Self-assessment is a learnt skill and requires practice.”
“Two areas that seem to be very important in aiding accurate self-assessment are the use of precise standards and the availability of feedback.”
Butterworth (2010) - Calibrating rating scales and rubrics
“Analytic global rating scales (where performance is broken into component parts that are scored individually first and then summed up to give an overally performance score) have been shown to be valid and reliable for assessing both students and physicians in practice.”
“Such scales need to be calibrated to ensure both tutors and students are using them accurately in the same way.”
“Familiarity with a tool through regular use and training is also important if it is to be used effectively.”
Butterworth (2010) - The role of feedback from peers and mentors
“Medical education and social cognition literature both clearly demonstrate that individual reflection is insufficient for accurate self-assessment.”
“Many studies expound the importance of peers or mentors for providing such support and feedback in an acceptable way to practicing physicians.”
“Feedback needs to be timely and given in a constructive way.”
Butterworth (2010) - Convergence (and reliability) of scores
“The working hypothesis of this study was that with practice and feedback each individual’s self-score would come closer to agreeing with the tutor score, demonstrating an improvement in ability to self-assess.”
A Tentative Framework/Model
36
A Tentative Framework/Model
37
A Tentative Framework/Model
2) Spiral through the 10 KEPPS Standards two full cycles by program completion (1st cycle-Content Knowledge goals; 2nd cycle-Professional Skills goals)
38
https://pixabay.com/photos/spiral-round-route-symbol-bent-2706905/
A Tentative Framework/Model
39
A Tentative Framework/Model
40
A Tentative Framework/Model
41
42
I-S-U-K-I-F : A Tentative Add-On to Self-Study CAEP Reporting
43
44
I-S-U-K-I-F
INFUSE SELF-RELIANCE
SPIRAL KEPPS STANDARDS
UTILIZE SELF-ASSESS RUBRIC
KEPPS ASSESSMENTS CONVERGE
INTERNAL LOCUS OF CONTROL
FOLLOW FOR CAEP R4.1A DATA
Ask Jeff Sadler at Fort Hays for Further Research
I talked to FHSU Mathematics instructor Jeff Sadler about this idea three or four years ago. He told me there is research suggesting that the most reliable data on how a person is doing comes from within that individual himself or herself, ideally. I think he may also have said it is a matter of how do we obtain that information – And now I hope that I can bring today’s suggestions to him and we can reassess the matter. But if it works, it would be a wonderful way to collect valid data on what CAEP calls Completer Effectiveness. Additionally, we would have equipped teachers for longer term job satisfaction and self-realization, ideally. :)
45