Person Reidentification

CS517 Mini-Project

Arnav Kharbanda(2021CSB1072)

Yashasav Prajapati(2021CSB1143)

References

• K. Zhou & T. Xiang, 2019
• L. Zheng, Y. Yang, & A. G. Hauptmann, 2016
• M. Xiong & Z. Gao, 2022
• Github - Code Implementation

Boston Marathon Bombing on April 15, 2013

• Two terrorists, brothers Dzhokhar Tsarnaev and Tamerlan Tsarnaev, planted two homemade pressure cooker bombs which injured hundreds of people and killed three.
• The investigators had to sit through hundreds of hours of CCTV footage and one of them had to watch the same segment of the footage more than 400 times.
• Later found, the suspects were already on the police records for under age drinking and other activities and were also enrolled in a University in Boston.

This investigation could’ve been faster, if they had the right technology.

This clearly shows the reason we need Person Re-identification

Problem Statement

Person re-identification is a computer vision task that involves identifying an individual in a series of images or video frames captured from one or more cameras. The goal is to match images of a person captured in different camera views or at different times, even if their appearance may have changed due to variations in lighting, pose, or occlusion.

Sample

Source: Science Direct

Source: Prid Webpage

INPUTS

Query Image

The person re-identification engine takes a single image as input and utilizes the dataset's gallery of images to identify all possible sets of images that may correspond to the same person.

ILLUSTRATION

INPUTS

Engine

OUTPUTS

Set of Images

• The output of the system is a set of true positive and false negative images.
• True positive images are those where the system correctly matches the image of an individual from the gallery with another image of the same individual from a different camera view.
• False negative images are those where the system fails to correctly match the image of an individual from the gallery with another image of the same individual from a different camera view.

ILLUSTRATION

OUTPUTS

Some More Illustrations

DataSets Used

• Market-1501
• Duke MTMC - reid

Evaluation Metrics Used

• Rank1 = total (most probable == actual true label) / total number of images

​

• Rank5 = total(top5 most probable label == actual true label) / total number of images

​

• Rank10 = total(top 10 most probable labels == actual true label) / total number of images

​

• Rank20 = total(top 20 most probable labels == actual true label) / total number of images

​

• mAP(Mean Average Precision) :

Mean of this over m queries gives mAP

What was the solution approach?

The general solution is a 2 step process.

We use deep learning to improve upon this technique.

1. Feature Extraction - Extracting features for every person images.
2. Feature Matching - Matching features to get a similarity score.

Progress in person Reidentification

Source: Person Reidentification Past, Present and Future

State of the art solutions - Metric Learning(uses deep learning)

Our current solution uses softmax loss or metric learning with triplet loss.

SoftMax Loss

​

Triplet Loss

• a = anchor image
• p = positive image
• n = negative image
• m= margin.

• C is the total number of classes,
• aⱼ denotes the output of the jᵗʰ neuron of the last fully connected layer before softmax activation,
• pⱼ denotes the softmax activation of the jᵗʰ neuron,
• xᵢ denotes the iᵗʰ image in a training batch of size M,
• tᵢ denotes the target one-hot encoded vector of image i,
• fᵢ be an encoding of xᵢ and it is also the input of the last fully connected layer,
• W is the weight of the last fully connected layer, (bias = 0 (let)),
• Lᵢ is the cross-entropy loss for image i, we have:

Works on the fact that if two images are similar, we bring them closer and if they’re not similar, we take them far apart

Using these losses in code to train a model

• Default model we used: Resnet50(pretrained)

• Total classes configured

• Loss(default -> softmax)

• Use Pre-trained model

• Use GPU

Using these losses in code to build the engine

Building Engine

Engine using SoftMax loss

Engine using Triplet Loss

Resnet50

Residual Network - A very powerful type of convolution neural network. It’s pretrained model is used in training of the person re-identification model using the losses discussed.

Result Evaluation Metrics Obtained from our training and testing

Comparison with NVIDIA Labs

Our Results

NVIDIA Labs Results(DG-Net NVIDIA’s own)

Almost similar results on Market-1501 dataset

ONE OF OUR RESULTS

Method Based

on Multi-Attribute Feature Generation

The proposed method consists of two main steps: multi-attribute feature generation and similarity score calculation. In the multi-attribute feature generation step, the input image is processed by multiple attribute classifiers, which generate features related to the color, texture, and shape of the person. Then, the features are combined into a single representation using a feature fusion method. In the similarity score calculation step, the Euclidean distance between the feature representations of the query and gallery images is calculated to obtain the similarity scores.

Alternate method

Source: A Lightweight Efficient Person Re-Identification Method

Comparison with Multi-Attribute Feature Generation Method

Our Results

MAFG Result

Learnings

• Importance of choosing appropriate distance metrics
• Metric learning can learn distance metrics that effectively measure similarity between images
• Choosing appropriate distance metrics can improve model performance
• Need for careful selection and annotation of training data
• Quality and diversity of training data impact effectiveness of metric learning
• Careful selection and annotation of training data can ensure the model learns useful and relevant distance metrics
• Potential for transfer learning
• Metric learning can be used to learn distance metrics for person re-identification across different domains and datasets
• Transfer learning can leverage knowledge learned from one domain to improve performance in another domain

​

Thanks