Fine Grained Vehicle Classification

IAM Metrics Project, Camera/LIDAR Fusion-Based Algorithm Team

By: Mayank Mathur

Mentor: Varun Jammula

EGR 598: Connected & Automated Vehicles (2022 Fall)

Objective

• The objective of the project was to work with the Camera/Lidar fusion base algorithm team to improve their car model and make detection.
• Create a dataset of monocular images of car models.
• Develop a classification model to identify car models using monocular images.
• The dataset generated by scraping Carvana website for vehicle models relevant for the team and can be later used by team for any other requirement.

​

​

​

​

Need for the Project

• The Camera/Lidar fusion team is working on vehicle localization and traffic scene reconstruction framework using infrastructure based camera systems.
• The paper they published can reliably localize vehicles and measure its velocity using monocular cameras mounted on road infrastructure.
• The system however cannot identify the model of the vehicle.
• Without the knowledge of vehicle model the framework cannot provide the full context of the traffic scene.

Details About Existing System

Fig. 2:Example of Reconstructed Traffic Scene

Fig 1. The CAROM Framework Architecture

Developed Solution

• Dataset for 10 vehicle models was generated by scraping Images from Carvana.
• The vehicle models selected were most common vehicle models present in the monocular camera footage being used by the team and overall most common vehicles on road.
• Deep learning models were trained to add the context of vehicle type and model.
• The model trained showed test accuracy of 98.7% and validation accuracy of 98.4%.

Dataset Description

• Carvana provides 360 view of multiple car models.
• These Multi POV images are loaded as a 4x8 image grid within the website and can be accessed by inspecting the source code.
• These image grids were then cropped into individual images of size 1920x1080 pixel images.
• Total images in dataset: 2,368

​

Fig. 3: Image grid from carvana

Dataset Description Contd.

Fig 4. Class distribution for training set

231, 288, 116, 231, 173, 288, 288, 288, 231

​

​

Fig 5. Class distribution for validation set

25, 32, 12, 25, 19, 32, 32, 32, 25

​

​

​

​

Model Description and Training Parameters

• Transfer learning was used on resnet-18 and resnet-50 models. Resnet-18 was finally selected as it is faster and the loss of accuracy from resnet-50 is not that high.
• Number of Epochs:20
• Learning Rate: 0.003
• ADAM optimizer was used.
• Activation function: Logsoftmax function
• Loss function: Negative Log Likelihood Loss(NLLLoss)
• The model was trained on google colab.

Fig 6. Train and Test Loss curve for resnet-18 model

Sample Predictions

Limitations and Future work

• Within the scope of this project:
• Test the model on real images from video and calculate real life accuracy.
• Outside the scope of this project:
• Object detection model like YOLO or FasterRCNN will offer both localization and multiclass predictions.
• More car models can be added to create a more meaningful model
• Other data sources can be used to improve training.
• Color detection

​

References

[1] CAROM - Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures,arXiv:2104.00893

[2] Carvana

[3] [1512.03385] Deep Residual Learning for Image Recognition

[4] PyTorch 1.13 documentation

​

​

Thank You