Veritas AI: Object Classification
Aarush Kumar, James Ellis, Jana Farid, Hargun Malhotra, Sylvie Yao
11/24/2024
Project Motivation
As the world population increases, the need for autonomous cars rises. However, not only do they need to function, they also need to be safe, particularly safer than humans…
How can we make autonomous vehicles as accurate as, or more accurate than us?
Project Motivation
Humans are much more safer than we all think:
Average Autonomous Vehicle Accuracy:
98.90%
Average Human Accuracy:
99.999%
11000 more crashes per million!
Although features such as Tesla Autopilot and Ford BlueCruise have revolutionized the car industry with their self-driving capabilities, and have been tested for years on end, they are not nearly as accurate as us.
This is why we have developed a recognition system to improve the reliability of electric cars:
We have programmed, tested, and ran multiple layers of NN/CNN’s, and have tested various activation functions as well as restructured entire frameworks for further experimentation.
We have used the CIFAR 10 Image Database to function Autonomous Accuracy better than humans.
CIFAR-10 stands for the Canadian Institute for Advanced Research 10-class dataset. This dataset is designed to test algorithms for object recognition and image classification tasks. It was created by Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton at the University of Toronto. The dataset was introduced as part of a publication in 2009, building on earlier datasets like the 80 Million Tiny Images dataset.
Input variables: These are images, each represented as a 32x32x3 (width x height x RGB channels)
Output variables : These are categorical variables, representing one of the 10 classes.
Amount of Data:
The dataset contains a total of 60,000 images, of which 50,000 are training images and 10,000 are testing images. Each image is labeled with one of 10 classes.
Example Data
Balanced Dataset
MLP Concept
Flat Vector
MLP Concept
Flat Vector
Baseline Model and Results: MLP
S - Situation
T - Task
A - Action
R - Result
Baseline Model and Results: MLP Accuracy
S - Situation
T - Task
A - Action
R - Result
Train Accuracy: 0.6200000047683716
Test Accuracy: 0.5498999953269958
Baseline Model and Results: Tinkering With Different Activations
S - Situation
T - Task
A - Action
R - Result
Total params: 299,264 (1.14 MB)
Trainable params: 99,754 (389.66 KB)
Non-trainable params: 0 (0.00 B)
Optimizer params: 199,510 (779.34 KB)
CNN Model
Baseline CNN Model
Model Summary
Base Model | Train Accuracy | Test Accuracy |
CNN Model | 65.42% | 70.18% |
Transfer Learning Results
Total parameters: 4,960,800
Activation Functions
ReLU vs Leaky ReLU
Any value that is less than 0, such as -2, gets zeroed out. Any value greater than one is able to continue (remains the same.
Any value that is less than 0, such as -2, does not get zeroed out. It is multiplied by Alpha, or 0.01 in this case.
It’s leaky, hence the name
Leaky ReLU in Convolutions Results
Train Accuracy: 0.7610599994659424
Test Accuracy: 0.6966000199317932
Regularization
Kernel Regularization
weight_decay = 0.0005
kernel_regularizer=regularizers.l2(weight_decay)
No Regularization
Regularization | Training Accuracy | Testing Accuracy | Difference |
Kernel Regularization | 70.25% | 65.34% | 4.91% |
No Regularization | 87.01% | 72.24% | 14.77% |
Epochs = 10
Baseline CNN Model
S - Situation
T - Task
A - Action
R - Result
No. of incorrect predictions: 2776
Transfer Learning
Transfer Learning Fully Connected Layer
Same Fully Connected Layers
Testing Different Base Models
Base Model | Train Accuracy | Test Accuracy |
ResNet50 | 96% | 75% |
MobileNet | 98% | 73% |
VGG16 | 9.8% | 10% |
Transfer Learning Results
Epochs = 30
Transfer Learning Tradeoffs
Base Model | Total Parameters | Trainable Parameters |
ResNet50 | 91 Million | 30 Million |
MobileNet* | 406 Million | 135 Million |
VGG16 | 55 Million | 18 Million |
*made use of a larger data set