Bias in Light-Based Sensors
Soap Dispenser
Chukwuemeka Afigbo / Twitter
Bias in Light-Based Sensors
Soap Dispenser
Pulse Oximeters
Infrared Thermometer
Facial Recognition
Plethysmography
Chukwuemeka Afigbo / Twitter
Cleveland Clinic, 2019
Grace Cary/Getty Images
Najibi, 2020
McDuff, 2020
Bias in Light-Based Sensors
Soap Dispenser
Chukwuemeka Afigbo / Twitter
Bias in Light-Based Sensors
Soap Dispenser
Pulse Oximeters
Infrared Thermometer
Facial Recognition
Plethysmography
Chukwuemeka Afigbo / Twitter
Cleveland Clinic, 2019
Grace Cary/Getty Images
Najibi, 2020
McDuff, 2020
Bias in Light-Based Sensors
Infrared Thermometer
Infrared Thermometer
Soap Dispenser
Pulse Oximeters
Plethysmography
Plethysmography
McDuff, 2015
Plethysmography
Captured Video
Clinical Finger Pulse Oximeter (GT)
Arbitrary units
Time
Plethysmography
Captured Video
Clinical Finger Pulse Oximeter (GT)
Arbitrary units
Time
Plethysmography
Captured Video
Pulse Oximeter Ground Truth - 96.0 BPM
Time
Arbitrary units
Cropped Face Frame
Waveform from Camera - 96.7 BPM
Time
Arbitrary units
ML
[Yu, 2019]
Plethysmography
Captured Video
Cropped Face Frame
Estimated (Blue) and GT (Green)
Time
Arbitrary units
Plethysmography
Captured Video
Cropped Face Frame
Estimated (Blue) and GT (Green)
Time
Arbitrary units
Plethysmography with Darker Skin Tones
Estimated Waveform from Camera - 74.2
Arbitrary units
Arbitrary units
Pulse Oximeter Ground Truth - BPM 83.1
Arbitrary units
Time
Time
Cropped Face Frame
Plethysmography with Darker Skin Tones
Cropped Face Frame
Estimated (Blue) and GT (Green)
Time
Arbitrary units
Plethysmography with Darker Skin Tones
Cropped Face Frame
Estimated (Blue) and GT (Green)
Time
Arbitrary units
10% more darker skin tones fail cardiac monitoring standards* than lighter skin tones.
* Failure occurs when the percent error is greater than 10% from ground truth.
Contributions of This Work
fairness tradeoff of RGB and radar
Contributions of This Work
fairness tradeoff of RGB and radar
RGB camera and radar for improving performance
and fairness
Contributions of This Work
fairness tradeoff of RGB and radar
RGB camera and radar for improving performance
and fairness
Radar-Plethysmography
0.25 mm�
This is how much your chest moves when the heart beats
https://bit.ly/3Ql8zDm
Radar-Plethysmography
https://bit.ly/3Ql8zDm
0.25 mm�
This is how much your chest moves when the heart beats
Radar-Plethysmography
Radar
Small Chest Vibrations
Radar (77 GHz)
has 10 µm range sensitivity
Radar-Plethysmography
Clinical Pulse Oximeter (GT)
Estimated from Radar
Time
Time
Arbitrary units
Arbitrary units
Radar Matrix
Time
Distance (Frequency)
Error-Bias Plot
1
2
1
2
0
Error-Bias Plot
HR Error (MAE, bpm)
1
2
1
2
0
Error-Bias Plot
HR Error (MAE, bpm)
1
2
1
2
0
Better
Error-Bias Plot
HR Error (MAE, bpm)
Bias (MAE, bpm)
1
2
1
2
0
Better
Error-Bias Plot
HR Error (MAE, bpm)
Bias (MAE, bpm)
1
2
1
2
0
Better
Better
Error-Bias Plot
HR Error (MAE, bpm)
Bias (MAE, bpm)
Radar
RGB
1
2
1
2
0
Better
Better
Fusing RGB and Radar
RGB Video
Radar Range Data
Fusion Architecture
time
Waveform
Ground truth
Estimated
PPG Loss
Beyond Bias in Plethysmography
HR Error (MAE, bpm)
Bias (MAE, bpm)
Radar
RGB
Our Naive
Fusion
1
2
1
2
0
Better
Better
Fusing RGB and Radar
RGB Video
Radar Range Data
Fusion Architecture
time
Waveform
Ground truth
Estimated
PPG Loss
Performance Loss!
Fusing RGB and Radar
RGB Video
Radar Range Data
Fusion Architecture
time
Waveform
Ground truth
Estimated
PPG Loss
Performance Loss!
Need to impose fairness constraint!
Fusing RGB and Radar
RGB Video
Radar Range Data
Fusion Architecture
time
Waveform
Ground truth
Estimated
PPG Loss
Error-Bias Plot
HR Error (MAE, bpm)
Bias (MAE, bpm)
Radar
RGB
Our Naive
Fusion
Fair Fusion
1
2
1
2
0
Better
Better
Error-Bias Plot
HR Error (MAE, bpm)
Bias (MAE, bpm)
Radar
RGB
Our Naive
Fusion
Fair Fusion
1
2
1
2
0
Better
Better
3% more dark skin subjects fail cardiac monitoring standard (compare with 10% for RGB)
Back to Darker Skin Tone Participant
Estimated (Purple) and GT (Green)
Time
Arbitrary units
Cropped Face Frame
Estimated HR = 81 BPM and GT HR = 83.1 BPM
Second Order Dataset Details and Results
91 Participants
Method | Performance (Fairness) | Fairness | |||
MAE | MAPE | RMSE | PCC (r) | T-Test (APE %) | |
RGB [Yu 2019] | 1.78 (2.22) | 2.35% (2.63%) | 5.26 (4.05) | 0.91 (-0.25) | 2.1, 12.2 |
Our RF | 2.18 (0.51) | 3.05% (0.69%) | 6.12 (0.85) | 0.89 (-0.13) | 5.1, 8.4 |
Our Fair Fusion | 1.12 (0.67) | 1.52% (0.79%) | 3.42 (1.44) | 0.95 (-0.10) | 1.1, 4.2 |
RGB Camera
Fusion
Radar
Towards Fair Computational Imaging
Towards Fair Computational Imaging
Pulse Oximeters
Thermometers
Facial Recognition
IR Sensors
Other devices
Towards Fair Computational Imaging
Other Axes of Bias
Body Mass
Error
Pulse Oximeters
Thermometers
Facial Recognition
IR Sensors
Other devices
Towards Fair Computational Imaging
Pulse Oximeters
Thermometers
Facial Recognition
IR Sensors
Other devices
Other Axes of Bias
Body Mass
Error
Completely Fair Devices
Error
Bias