U21BMP37�ROBOTICS IN MEDICINE

Dr. N. Rajasingam

UNIT -IV- IMAGE GUIDED INTERVENTIONS

Robot compatibility with medical imagers - Image segmentation and modelling - Tracking devices - Frames and Transformations - Surgical navigation.

​

CO4: Analyze the technical aspects of robotic technology in image guided interventions (Analyze)

Robot compatibility with medical imagers

• Image guidance is a common methodology of minimally invasive procedures.
• Depending on the type of intervention, various imaging modalities are available.
• Common imaging modalities are
• computed tomography
• magnetic resonance tomography
• ultrasound

Image Modalities

• Robotic systems have been developed to enable and improve the procedures using these imaging techniques.
• Spatial and technological constraints limit the development of versatile robotic systems.
• The main benefits of image-guided procedures in comparison to open or endoscopic surgery are reduced invasiveness and avoiding general anesthesia.

• The usage of computer-based systems to provide pre- and intra-operative imaging to perform minimally or non-invasive interventions has become the standard procedure in many medical fields. 
• The most common imaging modalities used to acquire the necessary pre-operative images are computed tomography (CT) and magnetic resonance tomography (MRI), due to their high spatial resolution and versatility.
• The utilization of ultrasound imaging (US), furthermore, allows for real-time intra-operative imaging due to its fast image acquisition. 

• The introduction of these procedures into the clinical domain, however also leads to the necessity of adapting to constrained working environments.
• Due to the missing direct sight into the situs, the surgeons have to ensure very high precision in handling the surgical tools, while relying on image data and in most cases tracked navigation systems. 

• To provide optimal assistance, the development of actuated manipulators and robotic systems received much attention.
• By co-registering these systems with medical imaging, it is possible to achieve higher accuracy and to alleviate highly complex procedures compared to manual performance.

CT-Guided Interventions

• The high spatial resolution of computed tomography (CT), especially of structures with a high density such as bone, makes this method one of the major imaging modalities in medicine.
• It is commonly used in head and neck, as well as lung interventions, for angiography and imaging of the axial skeleton and extremities.
• The fast image acquisition makes it a useful tool for intraoperative imaging.

• However, since CT imaging involves significant exposure of ionizing radiation for both patient and physician, the ratio of clinical benefit and risk must be taken into account.
• Therefore, in the last years, much effort has been made to introduce robotic systems that can operate under CT guidance, to alleviate the radiation exposition.
• The positioning and insertion of needles and cannulae to perform biopsies or therapeutic procedures is the prominent use case for CT-guided robotics.

MRI-Guided Interventions

• Magnet resonance imaging (MRI) enables image-guided interventions without exposing the patient to ionizing radiation.
• MRI technology provides arbitrary slice position, 3D images with variable soft-tissue contrast at near real-time speed.
• While MR imaging provides high-quality visual information during interventions, it has limitations for conventional as well as robot-assisted interventions.

• The spatial constraints inside the MRI bore limit the access to the patient during imaging.
• This also hinders utilizing robotic systems, as these need to fit in the residual space between the patient and the MRI.
• Furthermore, the high-strength magnetic field impedes the usage of conventional, metal-based materials for the robotic systems.
• To avoid degrading the image-quality, special considerations need to be taken when designing robotic systems for MRI-guided interventions.

​

• MR image-guided robot-assisted interventions i.e., cannula placement for biopsies, allow for precise positioning of the probe while ensuring tissue classification.
• The robot carries the potential to speed up the procedure thereby reducing the burden of the patient lying uncomfortably in the MRI.

• MRI-guided catheter-based interventions are an emerging field due to the advances in real-time MR-imaging and catheter tracking.
• The live-imaging enables faster procedures while allowing to monitor the instrument’s position.
• The interventionalist needs to interact with the MRI-scanner while performing the procedure.
• Thus, the controls and monitors need to be brought into the MRI-suite, or the procedure has to be performed from the control room. 

Robotic-Assisted US-Guidance

• In contrast to MRI- and CT-imaging, ultrasound imaging is widely available, because of its low cost and non-invasive imaging technique.
• Due to its properties, US imaging requires maintaining contact with the patient throughout the intervention.
• To ensure precise and reproducible imaging, robotic-assistance has been investigated to improve the imaging process or enable new intervention techniques.

• During manual US imaging, the interventionalist needs to provide the dexterity and sensitivity to position the US probe while incorporating the anatomical conditions.
• Thus, complex technical systems are necessary to automate the imaging process and also ensure patient safety.
• Active safety can be achieved through additional sensors. 

​

Medical imaging

• Medical imaging is the technique and process of creating visual representations of the interior of a body for clinical analysis and medical intervention, as well as visual representation of the function of some organs or tissues.

Computer Vision System

Levels of Processing

• Low-level processing
• Standard procedures are applied to improve image quality
• Procedures are required to have no intelligent capabilities.
• Intermediate-level processing
• Extract and characterize components in the image
• Some intelligent capabilities are required.
• High-level processing
• Recognition and interpretation.
• Procedures require high intelligent capabilities.

Architecture Modules

Image Acquisition

Digitization

• Digital image processing implies a discrete nature of the images.
• There are two steps in which it is done:
• Sampling: sampling rate determines the spatial resolution of the digitized image
• Quantization: quantization level determines the number of grey levels in the digitized image

Sampling and Quantization

​

• Resolution

​

​

• Grey Level

Preprocessing

• Enhancement
• Filtering
• Registration
• Calibration
• Transformation

Enhancement

• The objectives of image enhancement techniques is to process an image so that the result is more suitable than the original image for a specific application .
• Image enhancement techniques can be divided into two broad categories:
• Spatial domain methods .
• Point Processing
• Histogram Equalization
• Image Subtraction
• Frequency domain methods.
• Fourier transform

Filtering

• The objective of filtering is to remove noise.
• Techniques
• Averaging Filter
• Median Filter
• Max/Min Filter

Smoothing

• The aim of image smoothing is to diminish the effects of camera noise, spurious pixel values, missing pixel values etc.
• Two methods used for image smoothing.
• Neighborhood averaging
• Edge-preserving smoothing

Registration

• Medical image registration, for data of the same patient taken at different points in time such as change detection or tumor monitoring.
• Unimodal Registration: This term refers to the relative calibration of images that have been acquired with the same modality.
• Multi-Modal Registration: The images to be compared are captured with different modalities.

Transformation

• Image transforms can be simple arithmetic operations on images or complex mathematical operations which convert images from one representation to another.
• The transformation is intended to select the most prominent or relevant features.

Transformation

Image analysis

• It includes all the steps of processing, Like,
• Feature Extraction
• Segmentation
• Classification

Segmentation

• Medical image segmentation involves the extraction of regions of interest (ROIs) from 3D image data, such as from Magnetic Resonance Imaging (MRI) or Computed Tomography (CT) scans.
• Used to provide a precise and accurate representation of the objects of interest within the image, typically for the purpose of diagnosis, treatment planning, and quantitative analysis.

• Thresholding: involves setting a pixel intensity threshold to separate regions of interest from the background.
• Region Growing: starts from a seed point and expands the region by including adjacent pixels with similar characteristics, such as intensity or texture.
• Edge-Based Methods: identify boundaries between structures, allowing for segmentation by detecting rapid changes in pixel intensity.
• Model-Based Methods: uses mathematical models or statistical approaches to segment structures based on their expected properties.
• Machine Learning: can learn to identify structures from large datasets.

​

Feature Extraction

• Feature extraction is defined as the first stage of intelligent (high level) image analysis.
• It is followed by segmentation and classification
• Levels of Feature Extraction:
• Data level
• Region Level
• Texture Level
• Edge Level
• Pixel Level

Classification

• Statistic Classifiers
• Syntactic Classifiers
• Computational Intelligence-Based Classifiers
• Neural network
• Fuzzy Algorithm

​

Image Visualization

• It refers to all types of manipulation of this matrix, resulting in an optimized output of the image.
• It includes,
• Shading
• Display
• Reconstruction

Fusion Imaging

• Melting together images from different modalities to create a new (hybrid) image.
• Multimodal Fusion: Images of different modalities (PET, CT,MRI etc.)

Key aspects of Medical image segmentation

• Identification of Anatomical Structures
• Disease Detection and Diagnosis
• Treatment Planning
• Image-Guided Surgery
• Monitoring Disease Progression
• Customization of Treatment
• 3D Visualization

Medical Image Frames

• Spatial Frame: defines the physical dimensions and locations of structures within the human body. It establishes a coordinate system for the image, allowing for the precise localization of anatomical structures. Common spatial frames include the Cartesian coordinate system (x, y, z) and the polar coordinate system for specific applications.
• Temporal Frame: relates to the timing and sequencing of images, particularly in dynamic medical imaging, such as cardiac imaging or functional MRI. It enables the reconstruction of a series of images to visualize changes over time. In temporal frames, images are often acquired with specific time intervals.

• Patient Frame: refers to the positioning and orientation of the patient's body during image acquisition. It is essential for standardizing image orientation and comparisons. Radiological images are typically presented in the anatomical position, where the patient is upright, with the arms at the sides and the palms facing forward.
• Modality-Specific Frames: Different imaging modalities may have specific frames tailored to their requirements. For example, ultrasound images may have their own frames to account for probe orientation, while PET scans have frames for handling the 3D distribution of radioactive tracers.

Image Transformations

• Geometric Transformations: These include translation, rotation, scaling, and shearing, which can be applied to change the spatial orientation, size, and shape of an image. Geometric transformations are used to correct image misalignments or for image registration.
• Intensity Transformations: Intensity transformations involve changing the pixel values of an image. Common techniques include contrast enhancement, histogram equalization, and gamma correction, which improve the visual quality of images or highlight specific features.

• Frequency Domain Transformations: These transformations, such as the Fourier Transform or Wavelet Transform, analyze an image in the frequency domain. They are useful for tasks like filtering, detecting periodic patterns, and feature extraction.
• Spatial Domain Transformations: Spatial domain transformations are operations that directly modify pixel values in an image. Filtering, sharpening, blurring, and edge detection are examples of spatial domain transformations. They enhance or extract specific image features.

• Non-linear Transformations: These transformations involve more complex changes to the image, such as deformations to correct for anatomical variations, tissue warping, or distortion corrections in different imaging modalities.
• Image Registration: Image registration is a transformation process that aligns two or more images to match each other spatially. It is crucial for fusing images from different modalities or time points for analysis or treatment planning

Surgical navigation

• A method to guide surgery using medical images as a guidance map.
• The goal of surgical navigation is to maximize the treatment effect of surgery while minimizing trauma to patients by avoiding damage to critical structures.
• The most common form of surgical navigation is image-guided surgical navigation, which is mainly used to perform open surgery in a minimally invasive fashion.

• An advanced form of surgical navigation uses image-directed robots as guides or even tools to perform operative maneuvers for an operating physician. 
• The technologies that make surgical navigation possible include surgical planning, guidance workstations, tool tracking, and patient-to-image registration. 

• Anatomical and/or functional imaging modalities like computed tomography (CT), magnetic resonance imaging (MRI) and ultrasound, often combined with contrast agents, and molecular imaging modalities like single-photon emission computed tomography (SPECT) and positron emission tomography (PET) have become standard tools to aid in the diagnosis, monitoring and treatment of disease or injury.
• Yet, translating this wealth of detailed preoperative imaging information into better surgical treatment and clinical outcome is an ongoing challenge.

• Patient scans usually provide a 3D map of the disease, often placed in the context of the patient’s anatomy, that surgeons can use as a reference to guide them during an intervention.
• It would be very convenient for the surgeon to know exactly where surgical tools are on this map relative to the target location or, even better, to be provided with an optimal path from the tools towards the target.

​

• Navigation is a collective term that describes any workflow where patient scans, real-time tracking, and, occasionally, computer-aided planning are combined into real-time spatial information that provides orientation and sometimes even guidance to reach the target location during an intervention.
• The main benefit of this technology is the possibility to precisely indicate where structures of interest are located relative to the surgical tools in 3D.

• This is possible even when the structures of interest are covered by tissue and cannot be seen during surgery.
• Navigation promises to bring machine precision to clinical interventions, and will likely contribute to the emergence of more precise, less invasive and, hopefully, more effective procedures.

• The tracking systems are an essential component in all navigation workflows as they define the intraoperative coordinate system during an intervention.
• They are used to estimate the position and orientation of specially marked objects.
• These estimates, combined with registrations, enable the placement of tracked tools, patient scans and, if available, (computer-aided) planning in the same coordinate system.

Components of Computer-assisted Surgical Interventions

Navigation workflow using Optical tracking

• 1-Patient with tracker
• 2-multiple patient scans
• 3-computer-aided planning
• 4-tracker visible in preoperative and intraoperative coordinate systems
• 5-tracking system
• 6-navigation platform
• 7-tracked tools
• 8-patient with tracker on OR table

​

​

Navigation using Electromagnetic tracking

• For object tracking, there are a number of different techniques available, e.g. near-infrared (NIR) optical tracking, electromagnetic (EM) tracking, mechanical tracking and acoustic tracking.
• With NIR optical tracking systems, the emission and detection of NIR light is used to determine the position of trackers in space.
• To obtain stereo-vision, thus depth optical perception, this NIR light has to be captured by at least two cameras in a known spatial configuration.

• NIR optical tracking can only work when enough fiducial markers (small objects that each approximate a point and jointly representing three noncollinear points of a tracker) are in the line of sight of these NIR cameras.
• EM tracking systems, on the other hand, rely on variations in the magnetic field generated by a dedicated field generator to determine the position of sensor coils present on the tracker relative to the generator.
• The varying magnetic field induces current and potential in the coils.

​

• Usually, multiple coils are combined into a single tracker, and the combined readings from these coils provide enough information to estimate the position and orientation of the tracker in space.

Crucial Role

• Enhanced Precision
• Minimally Invasive Surgery
• Complex Procedures
• Real-time Feedback

Challenges

• Cost
• Learning Curve
• Integration

Limitations

• Technical Errors
• Line-of-Sight Challenges
• Radiation Exposure
• Invasive Setup

Patient Tracking

• In most navigation workflows, the purpose of tracking surgical tools is to determine their position relative to the patient’s anatomy and the diseased tissue therein as such to better guide the surgeon during the procedure.
• The typical navigation workflow will make use of preoperative imaging, meaning that the preoperative imaging data set has to be coupled to the interventional intraoperative coordinate system.

​

• The trick used to achieve this preoperative and intraoperative co- registration is to place a special tracker at the same position on the patient during preoperative imaging and during the intervention.
• This tracker is, by design, both visible to the tracking system and easily segmented from the preoperative scan.
• Once segmented, the position of the tracker relative to the patient can be calculated, leading to a registration between the coordinates of the patient and the tracker.

​

• To use the patient scan as a 3D map that is accurately positioned in the intraoperative coordinate system, the patient-to-tracker registration has to be coupled to the tracked position of the tracker in the intervention room.
• Essential requirements for precise registration are the identical placement of the tracker during preoperative imaging and during the intervention, and as little tissue deformation as possible between imaging and the intervention.

• A tracker is an object that is visible to the tracking system and holds enough information for the tracking system to unambiguously establish all six degrees of freedom of the tracker in 3D space (three degrees of freedom for the position and three for the orientation).
• Trackers can be attached to surgical tools or to portions of the patient’s anatomy, enabling their tracking.

​

• During the intervention, the position of the fiducials then had to be marked by a tracked pointer, thus providing the connection between patient scans and tracking system.

​

Tracking of Surgical Tools

• A similar, but simpler, approach is used to determine the position of the surgical tools in the intraoperative coordinate system. F
• or this, a tracker is attached to the surgical tool that needs to be navigated.
• Here it is crucial that this tracker is placed at a predefined position on the surgical tool, and that the tool is calibrated relative to the navigation platform.

​

• This calibration, in combination with the tracking information of the surgical tool tracker, allows tracking of the tip of the tool (or of any other part of the tool that is relevant during the intervention) thereby providing navigation from the perspective of the tip of the surgical tool.

​

AR‐guided distal interlocking procedures

Calibration of the EM transmitter with an optical marker by using a combined tool

Matrix transformation of AR‐guided distal interlocking procedures

Registration

• Connection between two coordinate systems (preoperative with patient scans and intraoperative with tracking information) via tracking of trackers, allowing objects from both coordinate systems to be shown in a single coordinate system.

​

Rigid registration: rotation and translation

Elastic registration: deformation field

Computer-aided planning

​

​

​

​

​

​

​

​

(a): raw patient scans

(b): segmentation of kidney lesion (yellow), nearby critical structures (orange) and nearby bones (pink)

(c): optimal paths to lesion (green dotted lines); as short as possible whilst avoiding damage to critical structures

​

Benefits of incorporating tracking devices in surgical procedures

• Enhanced Precision
• Minimally Invasive Surgery
• Improved Safety
• Reduced Radiation Exposure
• Better Visualization
• Enhanced Postoperative Outcomes
• Optimal Implant Placement