1 of 38

Characterizing Deep Learning Neural Network Failures between Algorithmic Inaccuracy and Transient Hardware Fault

Sabuj Laskar, Md Hasanur Rahman, Bohan Zhang, Guanpeng Li

2 of 38

Motivation

DNN has been increasingly deployed in many areas

Computer vision, NLP, autonomous vehicles (AVs)

DNN reliability becomes important

ISO 26262 safety standard requires no more than 10 FIT (10 failures in every 10⁹ hours)

3 of 38

Soft Error

[1] Design of Low-Cost Reliable and Fault-Tolerant 32-Bit One Instruction Core for Multi-Core Systems

Taken from [1]

Soft errors are inevitable!

4 of 38

Consequences of Error Propagation in DNNs

Single-bit fault^[2]🡪 Misclassification of image

[2] Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications, SC’17

Fault-free prediction label: Truck

Faulty predicted label: Bird

Reliability assessment: hardware vs software level

Software implemented fault injection (FI) simulation has lower cost

5 of 38

Previous Works Only Consider SDC

[2] Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications, SC’17

SDC

Cab

Bus

Albatross

6 of 38

Not All Misclassifications Are Equal

Misclassification

Cab

Bus

Similar action for AVs

Different action for AVs

Albatross

Misclassification

Need a different metric to differentiate safety critical misclassifications from non-safety critical ones

7 of 38

Our Hypothesis

Ground Truth

From Safety Critical Perspective of an AV

Small Deviation

(Bus -> Cab)

Significant Deviation

(Bus -> Albatross)

Intrinsic Algorithmic Inaccuracy

Silent Data Corruption

Safety concern of intrinsic algorithmic inaccuracies significantly lower than that due to SDC

DNNs need protection from SDC in safety critical situations

Misclassification

8 of 38

Existing DNN Reliability Measurement Tools

TensorFI^[3]

A fault injector for TensorFlow applications
Specifically, for TensorFlow 1 applications

TensorFI 2^[4]

A fault injector for TensorFlow 2 applications
This only supports sequential models

Sequential: VGG16, VGG19
Non-Sequential: ResNet50, ResNet101, GoogleNet, Xception, DenseNet121, DeseNet169, MobileNet

Most DNN models are non-sequential

[3] Tensorfi: A configurable fault injector for tensorflow applications, ISSREW’18

[4] https://github.com/DependableSystemsLab/TensorFI2

Need Support to inject faults in non-sequential DNN models with TensorFlow 2

9 of 38

Our Contributions

Developed open-source tool, TensorFI+, to support FI in non-sequential DNN models
Proposed new metrics to differentiate safety critical misclassifications from the perspective of AVs
Analyzed why DNNs need protection from SDC in safety critical situations

10 of 38

TensorFI+ Development

11 of 38

Keras Execution Flow Changes with TensorFI+

Operators’ structure changes in TensorFlow 2 are not allowed
Need Keras API for fault injection and propagation

Output (layer D) = KerasAPI(Destination layer D, Source layer S, Input values of S)
KerasAPI call to get output of target layer t
Random bit fip of output of layer t
Previous session gone, need API calls to propagate faulty output to final layer

Now we look at what changes in the execution flow is needed to support FI with TensorFI+
First of all, TensorFlow does not allow the operators’ structure to be modified once those tensor operators are constructed.
So, we leverages Keras APIs to inject fault and propagate the fault though layers.

KerasAPI needs the destination layer d, source layer s and input values of s to compute the output of a layer D.
Now we discuss the execution flow changes.
For example, we want to inject fault at target layer t.
So, we get the output of layer t from input layer by calling KerasAPI
We randomly flip a bit of the output of layer t
Because we stop the execution at layer t, the previous session containing the outputs of other visited layers are no longer there.
So, we call Keras API several time to propagate the fault from t to the final layer of DNN model.

12 of 38

FI in a Sequential Model

Compute layer 4 output from layer 1

Target layer for FI

Inject fault

KerasAPI(n,5, faulty output of layer 4)

13 of 38

Issues in FI in Non-sequential Model

Target layer for FI

Fault propagation

Missing output of layer 3 due to session lost during FI

Solution: call KerasAPI to get output of layer 3 first

KerasAPI(n,5, faulty output of layer 4)?

14 of 38

Solution: Super Layer

Super layers are not part of any branch
Any layer after a super layer is not dependent on any layer prior to super layer

No need to consider layers after 6 if we inject fault in layer 3

15 of 38

Simulation of FI with TensorFI+

Immediate next super layer

MDict






Layer 2

Immediate previous super layer

16 of 38

Simulation of FI with TensorFI+

Immediate previous super layer

Immediate next super layer

MDict





Layer 4
Layer 2

17 of 38

Simulation of FI technique of TensorFI+

Immediate next super layer

MDict





Layer 4
Layer 2

Immediate previous super layer

18 of 38

Simulation of FI technique of TensorFI+

Immediate next super layer

MDict




Layer 8
Layer 4
Layer 2

Immediate previous super layer

19 of 38

Simulation of FI technique of TensorFI+

Immediate next super layer

MDict




Layer 8
Layer 4
Layer 2

Immediate previous super layer

20 of 38

Simulation of FI technique of TensorFI+

Immediate next super layer

MDict



Layer 6
Layer 8
Layer 4
Layer 2

Immediate previous super layer

21 of 38

Simulation of FI technique of TensorFI+

Immediate next super layer

MDict


Layer 9
Layer 6
Layer 8
Layer 4
Layer 2

Immediate previous super layer

22 of 38

Simulation of TensorFI+

Immediate next super layer

MDict

Layer 10
Layer 9
Layer 6
Layer 8
Layer 4
Layer 2

Finally Compute the output of layer 12 using

only one KerasAPI(12, 10, inputs(10)) call

Immediate previous super layer

👉

23 of 38

Metrics to Differentiate Safety Critical Misclassification

24 of 38

Overview: Steps to Define Our Metrics

Create several groups based on similarity of objects
Organize all the groups into two supergroups based on safety concern
Define two metrics to measure whether a misclassification is safety critical or not.

25 of 38

Group Formation

Ball player

Scuba driver

Cab

Bus

Bullfrog

Goldfish

Baseball

Binoculars

Person

Four Wheeler

Small Animals

Tools and Household Chores

Groups

Objects

26 of 38

Organize Groups into Two Supergroups

Person

(Ball player, Scuba driver)

Four Wheeler

(Cab, Bus)

Small Animals

(Bullfrog, Goldfish)

Tools and Household Chores

(Baseball, Binoculars)

Safety Critical

Non-Safety Critical

Safety Critical

Non-Safety Critical

Super Group A

Super Group B

The second step is to organize groups into two supergroups
Initially we have two supergroups.
We rank Supergroup A as safety critical and supergroup B as non-safety critical.
We placed Person and Four Wheeler into supergroup A because any misclassification between Person and Four wheeler are safety critical.
Similarly we placed Small animals and Tools and Household chores in supergroup B because misclassification between them is not safety critical.
Here, an interesting fact is when ground truth is from supergroup A and prediction is from supergroup B, then it is safety critical. Because misclassifying a person as a small animal will result in a safety critical action.
But vice versa is not safety critical. Because predicting a baseball as a cab might cause activating the brake for the AV, but it won’t crate a safety concern.

27 of 38

Metrics: SCM and Non-SCM Probability

Safety Critical Misclassification(SCM) Probability

Original label is in Supergroup A and the predicted label is in Supergroup B
They are from different groups within Supergroup A

Non-Safety Critical Misclassification(Non-SCM) Probability

Non-SCM probability complements to SCM probability
They add up to 100%.

28 of 38

Benchmark & Experimental Setup

Demonstrated on 30 popular DNN models

VGGNets, ResNets, DenseNets

2 open-source widely used datasets

CIFAR-100, ImageNet

3000 random fault injections per DNN model
Measured SDC, SCM and Non-SCM probability in the evaluation

29 of 38

Results: SDC rates

Dataset	Model	Top-1 Accuracy	SDC Rate
ImageNet	VG16(Sequential)	71.18%	3.53%
	ResNet50(Non-sequential)	74.76%	1.43%
	DenseNet121(Non-sequential)	75.04%	1.20%
CIFAR-100	VGG19(Sequential)	71.53%	1.23%
	GoogleNet(Non-sequential)	76.70%	1.57%
	Xception(Non-sequential)	77.96%	2.00%

SDC rates range from 0.53% to 2.07% (error bars range from 0.10% to 2.95%)

across different non-sequential DNN models

30 of 38

Results: Fault Free Inference

CIFAR-100

ImageNet

SCM Probability of CIFAR-100 and ImageNet are less than 20% and 10% respectively

31 of 38

Results: FI in Correctly Classified Images

CIFAR-100

ImageNet

SCM probability of CIFAR-100 and ImageNet is around 30-40% and 10-38% respectively

32 of 38

Results: FI in Misclassified Images

CIFAR-100

ImageNet

SCM probability of CIFAR-100 and ImageNet is around 20-50% and 14-36% respectively

33 of 38

Conclusion

Built a FI tool, TensorFI+, for both sequential and non-sequential DNN resilience evaluation
We introduce two new metrics to differentiate safety critical misclassifications.
SCM probability is much higher with FI compared to fault free inference

Shows the necessity of protecting DNN models from SDC.

Our code is open source at https://github.com/sabuj7177/characterizing_DNN_failures

Sabuj Laskar

University of Iowa

34 of 38

Backup Slides

35 of 38

Why Keras (TensorFlow 2)

Primary ML software tool used by top-5 teams on Kaggle in each competition in the last two years

✓

Source: https://keras.io/why_keras/

36 of 38

Keras (TensorFlow 2) Execution Flow

Keras execution flow without TensorFI+

37 of 38

Fault Injection Methods

Randomly sampled 10000 images from all the datasets

Maintained similar distribution of Super Group A and Super Group B as the original dataset

Ran fault free inference and computed SCM and Non-SCM probability
Injected 3000 random faults in both correctly classified and misclassified images and computed SCM and non-SCM probability

38 of 38

Super Group Formation