1 of 21

2 of 21

3 of 21

4 of 21

5 of 21

6 of 21

ORIME CARDS

7 of 21

Characteristics of Effective Feedback

Specificity – Feedback is most effective when it is specific to the task being observed

Frequency – The more often feedback is given the more effective it is

Timing – Feedback is usually most effective when given in close proximity to the observed event/behavior�Positive / Negative – Positive feedback is as important as corrective feedback to reinforce effective behaviors�Learner Reaction – Give the learner time to react to the feedback�Action Plan – Feedback is most effective if the learner knows what to do for “next time”

Milestones within RIME

6 months 12 months 12-18 months 24 months

Reporter Interpreter Manager Educator

8 of 21

Neurology

9 of 21

Neurology

O

R

I

M

E

Observer – a novice in the clinical setting�Inaccurate or incomplete in the acquisition of patient information��Reporter – an expert organizer and conveyor of information�Obtains & clearly communicates the subjective information pertinent to the clinical condition�Obtains & clearly communicates vital signs, physical exam findings, labs, cultures, radiographs�Reads all notes and able to report accurately�Follows up on new labs/studies/findings throughout the day� Answers “what” or “factual” questions: “What were the MRI results?”; “What are the common causes of dizziness?”��Interpreter – an expert in diagnostics�Prioritizes problems from most important to least important in the assessment�Offers at least 3 reasonable differential diagnoses for new problems�Provides accurate analysis of exam findings and test results�Answers “why” or “clinical reasoning” questions “Why is the patient weak?”; “Why is the patient confused?��Manager – an expert in management�Consistently proposes a reasonable plan of care�Attempts to tailor the plan to the specific patient’s circumstances and preferences: “We should avoid benzos because of the patient’s instability”�Answers “How” questions – “How do we choose another blood pressure medication to get this patient to target?”��Educator – an expert in evidence based management and teacher�Poses thoughtful clinical questions and independently seeks answers�Uses reliable, applicable evidence based sources (guidelines, scales) to base clinical decisions that are pertinent to the patient�Shares new learning with other members of the team

10 of 21

Surgery

11 of 21

Surgery

O

R

I

M

E

Observer – a novice in the clinical setting�Inaccurate or incomplete in the acquisition of patient information��Reporter – an expert organizer and conveyor of information�Obtains & clearly communicates the subjective information pertinent to the clinical condition�Obtains & clearly communicates vital signs, physical exam findings, labs, cultures, radiographs�Reads all notes and able to report accurately�Follows up on new labs/studies/findings throughout the day� Answers “what” or “factual” questions: “What were the HIDA scan results?”; “What are the indications for cholecystectomy?”��Interpreter – an expert in diagnostics�Prioritizes problems from most important to least important in the assessment�Offers at least 3 reasonable differential diagnoses for new problems�Provides accurate analysis of exam findings and test results�Answers “why” or “clinical reasoning” questions “Why is the patient dyspneic?”; “Why is the patient distended?��Manager – an expert in management�Consistently proposes a reasonable plan of care�Attempts to tailor the plan to the specific patient’s circumstances and preferences: “I have to decrease the IV fluids and order lasix as the patient is reversing his inflammatory fluid shift and developing pulmonary edema.”�Answers “How” questions – “How do we decide when it is time to operate for a bowel obstruction”��Educator – an expert in evidence based management and teacher�Poses thoughtful clinical questions and independently seeks answers�Uses reliable, applicable evidence based sources (guidelines, scales) to base clinical decisions that are pertinent to the patient�Shares new learning with other members of the team

12 of 21

Faculty Feedback

Resources

Videos

Workshops

Facilitators

13 of 21

14 of 21

15 of 21

16 of 21

Trainees

Formative Evaluation

Direct Observation Mobile App

Summative Evaluation

Clerkship Management System

Evaluator

Observer Reporter Interpreter Manager Educator

Review formative evaluations. Select ORIME summative grade for trainee.

DO Request

!

Encounter

History

Presentation

Other Activities

Examples of Clinical Performance

Clerkship Timeline

Clerkship Evaluation Process

Rate level of supervision required.

Direct Observations

Welcome! This brief video will describe the VCU School of Medicine Clerkship Evaluation process. /1/ In the medical curriculum, there are eight core clerkships and each one can be as few as four and as many as eight weeks. Within this clerkship timeline, evaluators /2/ - which can be faculty, residents, preceptors, and others - will have several opportunities to evaluate a trainees’ /3/ clinical performance /4/ using innovative summative and formative evaluation tools. /5/ Evaluations begin once a trainee makes a request using the VCU direct observation mobile app. /6/ Evaluators will receive a notification on their phone to evaluate a trainee’s clinical performance, /7/ such as their ability to accurately conduct a history and physical exam, or effectively present a patient case. /8/ Each direct observation will rate the level of supervision required /9/ using the Ottawa and Chen scales. Narrative comments may also be submitted.

Once these direct observations are collected over the course of the clerkship, /9/ each evaluator can review their scores and notes for each trainee. Using all of these data, /10/ the evaluator will then decide which level of the ORIME framework the student has achieved.

Once selected, the evaluator will submit /11/ a summative evaluation for each trainee to the VCU clerkship management system or CMS. Unlike the Direct Observation formative tool, CMS is best accessed from a desktop or laptop computer.

The remaining portion of this video will define the ORIME framework, describe how to give behavioral feedback, and provide several examples of ORIME student performance in select clerkships.

17 of 21

Encounter

History

Presentation

Evaluator

Trainees

Formative Evaluation

Direct Observation Mobile App

Summative Evaluation

Clerkship Management System

Clerkship Timeline

Evaluator

Rate level of supervision required.

Observer Reporter Interpreter Manager Educator

DO Request

Review formative evaluations. Select ORIME summative grade for trainee.

!

Other Activities

Examples of Clinical Performance

Direct Observations

18 of 21

Formative Evaluation

Direct Observation Mobile App

Summative Evaluation

Clerkship Management System

Trainees

19 of 21

Formative Evaluation

Direct Observation Mobile App

Summative Evaluation

Clerkship Management System

Trainees

Welcome to the VCU School of Medicine evaluation process for clerkship evaluators. Trainees at the VCU School of Medicine receive formative and summative feedback as they progress through each clerkship and this feedback is used to document and improve their performance behaviors, and provide them a final clerkship grade.

Each clerkship has a start and finish time, as few as four and as many as eight weeks. Within this clerkship timeline, evaluators will have several opportunities to evaluate their trainees. An evaluation will begin once a trainee makes a request using the VCU direct observation mobile app. Evaluators will receive a notification on their phone to evaluate a trainee’s clinical performance, such as collecting a patient history or presenting a patient. Each observation must rate the level of supervision required using the Ottawa and Chen scales. Narrative comments may also be submitted. The VCU direct observation mobile app allows for a chronological log to be collected about a students clinical performance.

Once these direct observations are collected over the course of the clerkship, each evaluator can review the scores and notes they have submitted for each trainee, and then decide which level of the ORIME framework the student has achieved.

Information about ORIME and the ORIME criteria for each clerkship is available here.

Once the evaluators has reached a decision, the evaluator will then submit a summative evaluation to the VCU clerkship management system. This innovative app is best accessed from a desktop or laptop computer.

20 of 21

Encounter

History

Presentation

Evaluator

Trainees

Formative Evaluation

Direct Observation Mobile App

Summative Evaluation

Clerkship Management System

Clerkship Timeline

Evaluator

Rate level of supervision required.

Observer Reporter Interpreter Manager Educator

DO Request

Review formative evaluations. Select ORIME summative grade for trainee.

!

Other Activities

Examples of Clinical Performance

Direct Observations

Clerkship Evaluation Process

Welcome! This brief video will describe the VCU School of Medicine Clerkship Evaluation process. //There are eight core clerkships and each one can be as few as four and as many as eight weeks. Within this clerkship timeline, evaluators // - which can be faculty, residents, preceptors, and others - will have several opportunities to evaluate a trainees’ clinical performance // using innovative summative and formative evaluation tools. Evaluations begin once a trainee makes a request // using the VCU direct observation mobile app. Evaluators will receive a notification on their phone to evaluate a trainee’s clinical performance, // such as their ability to accurately conduct a history and physical exam, or effectively present a patient case. // Each direct observation will rate the level of supervision required using the Ottawa and Chen scales. Narrative comments may also be submitted.

Once these direct observations are collected over the course of the clerkship, each evaluator can review their scores and notes for each trainee. Using all of these data, // the evaluator will then decide which level of the ORIME framework the student has achieved.

Once selected, the evaluator will submit // a summative evaluation for each trainee to the VCU clerkship management system or CMS. Unlike the Direct Observation formative tool, CMS is best accessed from a desktop or laptop computer.

The remaining portion of this video will describe ORIME framework and provide several examples of ORIME performance in select clerkships.

21 of 21

Clerkship Evaluation Process

Welcome! This brief video will describe the VCU School of Medicine Clerkship Evaluation process. //There are eight core clerkships and each one can be as few as four and as many as eight weeks. Within this clerkship timeline, evaluators // - which can be faculty, residents, preceptors, and others - will have several opportunities to evaluate a trainees’ clinical performance // using innovative summative and formative evaluation tools. Evaluations begin once a trainee makes a request // using the VCU direct observation mobile app. Evaluators will receive a notification on their phone to evaluate a trainee’s clinical performance, // such as their ability to accurately conduct a history and physical exam, or effectively present a patient case. // Each direct observation will rate the level of supervision required using the Ottawa and Chen scales. Narrative comments may also be submitted.

Once these direct observations are collected over the course of the clerkship, each evaluator can review their scores and notes for each trainee. Using all of these data, // the evaluator will then decide which level of the ORIME framework the student has achieved.

Once selected, the evaluator will submit // a summative evaluation for each trainee to the VCU clerkship management system or CMS. Unlike the Direct Observation formative tool, CMS is best accessed from a desktop or laptop computer.

The remaining portion of this video will describe ORIME framework and provide several examples of ORIME performance in select clerkships.