Federated Learning of Medical Image Reconstruction (FLoMIR)

Semester 2 Project Plan

Yash Jani, Tanuj Kancharla, Izzy MacDonald, and Joshua Sheldon

Advisor: Dr. Mitra

Background

• For a SPECT scan a patient is injected with a tracer, and then scanned with gamma cameras.
• The image then has differing levels of brightness representing the amount of tracer found in that part of the body, captured in different angles.
• These images are then reconstructed so that medical professionals can easily read them.
• However, these algorithms are slow

Goal

• Develop a machine learning model that can perform the exact reconstruction much quicker than traditional methods
• Improve the accuracy of the existing training model through
• More realistic data
• Federated Learning (FL)

Approach #1

More Realistic Data

• Drastically improve on the quality and diversity of the previous dataset by improving the data generation pipeline:
• Improving the physics simulation
• Correcting the data dimensionality
• Introducing infractions in the heart
• Modifying the machine learning pipeline to train with the new, highly realistic synthetic data

Approach #2

Federated Learning (FL)

• Training the model with FL
• Enables training with real medical data from contributors while ensuring data privacy
• Realized through two applications: orchestrator, (server) and contributor (client)

Novel Features

• Applying FL is not novel, but our approach is largely focused on using proven techniques in a new problem space
• We believe we are the first to train a machine learning model to reconstruct medical images from SPECT scans, using synthetic data with artificially introduced heart infractions

Algorithms/Tools

• Machine Learning Model
• Pytorch
• Federated Learning
• React
• Flower
• Flask

• Data synthesis pipeline
• Python and Bash scripts
• XCAT Phantom Program
• XCAT+
• GATE

Technical Challenges

Integration

• Have the frontend respond to changes in the backend
• Having our FL applications successfully operate completely separate applications from the Flower framework
• Integrating our components into one reliable system.

Evaluation

• We should reliably be able to perform FL with our applications.
• That reliability should hold when performing FL with one orchestrator and four contributors.
• The accuracy of the machine learning model produced by our applications should be greater than the accuracy of a machine learning model trained on any one contributor’s data.

FLoMIR System Architecture

Progress Summary

Milestone

4

• Flower framework code
• Contributor application UI
• New ML model dimensionality
• Implement connection between the orchestrator application frontend and backend
• Implement secure communications between the orchestrator and contributor applications
• Run data generation pipeline on A.I. Panther

Milestone

5

• Finish implementation and testing of Flower framework
• Integration between Flower framework code and Federated learning applications
• Improve machine learning model accuracy
• Connect the contributor application’s frontend and backend

Milestone

6

• Perform a full system test and demo
• Compare full system performance to single-client performance
• Create user/developer manual
• Create demo video

Milestone 4 Task Matrix

Milestone 4 Task Matrix

Questions?

Credits: Sides from Slidesgo