AI in surgical training

Mark Coleman

Mark Coleman

Consultant Surgeon

Clinical Director

Associate Professor

Colorectal Unit

University Hospitals Plymouth NHS Trust

Disclosures

• Co-director Lapco International Consultancy 2017-present

​

• National Clinical Lead Lapco 2008-2014

​

• No influence on published work

Define AI

The capability of a computer to perform actions analogous to the human brain (McCarthy 1956)

Teenage sex

Becoming an expert surgeon

motivation

training

practice

persistence

awareness

Kind Thanks to Roland Valori and Robin Kennedy for this slide

Effective Health Care 1999;5(1)

persuasion

facilitation

consciously expert

unconsciously inexpert

Trainee

Trainer

SYSTEM

Becoming an expert surgeon

motivation

training

practice

persistence

awareness

Kind Thanks to Roland Valori and Robin Kennedy for this slide

Effective Health Care 1999;5(1)

persuasion

facilitation

consciously expert

unconsciously inexpert

Trainee

Trainer

SYSTEM

AI

AI uses in surgical training

AI in training - ?’s

AI & training

Priorities for AI in training

Competency evaluation

Objectives of surgical training

Basic operative AI metrics

Confirming Objective AI metrics in training

AI in RARP – DaVinci + DV logger 100 cases

• Instrument time
• Instrument path length
• Velocity of instrument
• Frequency of master controller clutch use
• Energy use
• Camera use
• Third arm swap

results

results

results

Results – 3D instrument path tracings

DEVELOPING BASIC HAND-EYE COORDINATION SKILLS�FOR LAPAROSCOPIC SURGERY USING GAZE TRAINING�Mark Wilson*, Mark Coleman† and John McGrath – Department of Urology, Royal Devon and Exeter NHS Foundation Trust and *Department of Sports and Health Sciences, University of Exeter, Exeter; and †National Training Programme in Laparoscopic Colorectal Surgery (LAPCO), Plymouth, UK�Accepted for publication 12 January 2010 �© 2010 THE AUTHORS JOURNAL COMPILATION © 2010 BJU INTERNATIONAL��Novices see instruments, experts see the field�Gaze analysis can be used for training ��

Conclusions - 1

Can distinguish between experts and novices for real-time events and kinematics

Surgical competence

• In surgery, competence is the ability to successfully apply professional knowledge, skills, and attitudes to new situations as well as to familiar tasks. A critical step in assessing surgical competency is developing methodology for competency evaluation and certification.

​

​

Is Competency Assessment at the Specialist Level Achievable? A Study for the National Training Programme in Laparoscopic Colorectal Surgery in England�

• Miskovic, Danilo MD, PhD, FRCS^*; Ni, Melody PhD^*; Wyles, Susannah M. MBBS, MSc, MRCS(Eng)^*; Kennedy, Robin H. MD, FRCS^†; Francis, Nader K. PhD, FRCS^‡; Parvaiz, Amjad FRCS, FRCS(Gen)^§; Cunningham, Chris MD, FRCS^¶; Rockall, Timothy A. MD, FRCS^‖; Gudgeon, Andrew M. MBBS, FRCS^**; Coleman, Mark G. MD, FRCS^††; Hanna, George B. PhD, FRCS^*
• Author Information Annals of Surgery: March 2013 - Volume 257 - Issue 3 - p 476-482

​

GAS FORM EXAMPLE

GRADING OF STEPS

1- 6 SCALE

GAS Form data – raw – vs CUSUM

Author Information Annals of Surgery: March 2013 - Volume 257 - Issue 3 - p 476-482

CUSUM plot – 3 trainees & average scores

Author Information Annals of Surgery: March 2013 - Volume 257 - Issue 3 - p 476-482

GAS form scores – procedural steps

Author Information Annals of Surgery: March 2013 - Volume 257 - Issue 3 - p 476-482

GAS form scores – LCS procedures

Author Information Annals of Surgery: March 2013 - Volume 257 - Issue 3 - p 476-482

�

Competency Assessment�Observational clinical human reliability analysis (OCHRA) (Concurrent Validity)

Miskovic et al Surg Endosc 2012 Mar;26(3):796-803

Competency Assessment�Reliable prediction of competent performers (Concurrent validity)

Miskovic et al Surg Endosc 2012 Mar;26(3):796-803

Sign off and outcomes

• Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program

Danilo Miskovic, Susannah M Wyles, Fiona Carter, Mark G Coleman, George B Hanna. Surgical endoscopy 2011, 25 (4) 1136-1142

​

Lapco sign off: risk factors for failure�

​

• Incomplete video (OR=8.7)
• Distal division vasc pedicle (OR=2.9)
• Inadequate resection (OR=5.8)
• SB Injury (OR=4.8)
• Damage to mesentery (OR=3.4)
• Unclear pedicle plane (OR=4.1)
• Out of dissection plane (OR=3.8)
• Poor anastomosis (OR=11.6)

Competency Assessment�Clinical Outcomes Related to LCAT Score (Predictive Validity)

Lapco Competency Assessment – Lessons Learned

• Competency assessment is an integral part of a training programme
• It identifies under-performers and difficult procedures
• It ensures high operative standards are achieved

​

• Educational research enables design and validation of assessment tools

​

• Assessment of consultants is feasible and acceptable

​

• High technical standards improve clinical outcomes

Does performance correlate to outcomes & can this be measured objectively?

• 20 videos of lap bypass from different surgeons – self selected
• Edited
• Critical components of each procedure
• Peer reviewed
• Anonymous
• Validated OSATS tool (objective structured assessment of technical skills)
• Gentleness, tissue exposure, instrument handling, time and motion
• Rated 1-5 (1 = resident, 5 = expert/master)

• Outcomes
• Any significant complication
• Reoperation
• 30-day mortality
• Readmission
• ED visits

• 10 reviews of each video
• Mean score
• No significant rating bias

• Scores 2.6-4.8
• 4 quartiles
• Bottom quartile mean score 2.9
• Top quartile mean score 4.4
• Surgeons in bottom quartile took 40% longer and perform fewer cases

​

Conclusions - 2

Surgical skills correlate to clinical outcomes……but do they longterm?

� J Clin Oncol 2010 10;28(2):272-8.� doi: 10.1200/JCO.2009.24.1448.

• Complete mesocolic excision with central vascular ligation produces an oncologically superior specimen compared with standard surgery for carcinoma of the colon

​

New Lapco Digital Learning Platform

Aim: to create a versatile user-friendly digital training platform for surgeons

Protect- Commercially Sensitive. Copyright Lapco Consultancy Ltd 2021

• Faculty (administrators) can build and manage their Lapco TT courses with automated course design and booking features.

​

• Other types of courses can be integrated on the system.

​

• Courses can be promoted across the user network.

​

Protect- Commercially Sensitive. Copyright Lapco Consultancy Ltd 2021

• Practical, quick and practical assessment tools to record and evaluate trainee and trainer performance at all stages of the pathway.

​

• Fully integrated communication between the trainee and assessor.

​

• Dashboard to evaluate data and compare results.

​

• Integrate (or develop) bespoke tools to assess training, clinical and economic outcomes.

​

Protect- Commercially Sensitive. Copyright Lapco Consultancy Ltd 2021

​

• Trainers (attendees) receive course materials directly to their account to access at any time.

​

• Attendees complete ‘Course Assessment form’ to automatically generate their certificate.

​

• Course feedback recorded to maintain programme quality.

​

​

summary

Thank you!

mcoleman1@nhs.net

@lapcoleadmark

+447789873190