Shoplifting recognition from video surveillance cameras

02

04

03

01

Table of contents

Dataset

Data Preprocessing

Models

Best Model

“You can have all of the fancy tools, but if [your] data quality is not good, you’re nowhere.” — Veda Bawo, director of data governance, Raymond James

“Without clean data, or clean enough data, your data science is worthless.” — Michael Stonebraker,

Adjunct Professor, MIT

Dataset

01

Data! Data! Data! I can’t make bricks without clay!

Dataset Summary

565

436

129

Non Shoplifter

Shoplifter

The duration of each video is between 10 to 18 seconds with fps = 25

Total Number of Videos

Data Preprocessing

02

If you torture the data long enough, it will confess.

How to Preprocess a video?

A video is?

Pre-�processing

• A Bunch of photos (frames) stacked together, which makes a video.
• Those frames are taken at a specific rate, like 25 frames per second.
• As the frames increase, the video becomes more smooth.

• To feed a video to a model:
• We take some frames from the video, then stake them together and feed them to a model.
• The frames then can be sampled randomly from the video or in a specific way you prefer.
• Also, we need to encode the classes from strings to integers

Sampling Frames

Randomly sample N frames from each video

Sample N frames from each video with a uniform distribution

Sample N frames from each video while giving more importance to middle frames

Subsampling in the temporal dimension

​

Preprocessing the Frames

We need to preprocess our sampled frames to match the input size of the model

Minor Class ?

Now we are ready to start training our models!

Models

03

Used Models

A 3D convolutional neural network architecture designed for video understanding tasks.

A video understanding model based on the Multiscale Vision Transformer architecture.

More formally, using 2D convolutional backbone along with Recurrent units to capture temporal relations

Our Strategy

Hybrid 2D Approach

• Use a pretrained CNN backbone for spatial processing.

​

• We choose MobileNetV2 for its speed and high accuracy.

​

• The CNN is distributed over time dimension with the TimeDistributed Layer.

​

• Sequence Learning via Recurrent Units like Bidirectional LSTMs

​

• A multilayer preceptron for final classification.

Hybrid 2D Approach

​

The Whole architecture

Hybrid Model Performance

Precision�0.75 %

Recall�14.28%

Didn’t stand for a lot of time

Accuracy�54.76%

R3d model

• The R3D model, short for ResNet-3D, is a convolutional neural network architecture specifically designed for video understanding tasks.

​

• It extends the successful ResNet architecture, which was originally developed for image classification, to handle spatiotemporal data found in videos.

R3d Model Key Features

R3d Model Performance

Precision�100 %

Recall�53%

Accuracy�76%

First place for a short of time

What is going on with the recall ?

We have zero false positives and very high false negatives. The model never says a positive if its not sure of it, and it says a negative easily !

​

​

MViT Model

• A video understanding architecture based on the Vision Transformer (ViT) architecture, extended to handle spatiotemporal data present in videos.

​

• “The benefit of this design is the presence of strong implicit temporal bias in video multiscale models. Authors show that vision transformer models trained on natural video suffer no performance decay when tested on videos with shuffled frames. This indicates that these models are not effectively using the temporal information and instead rely heavily on appearance. In contrast, when testing our MViT models on shuffled frames, we observe significant accuracy decay, indicating strong use of temporal information.”

MViT Model Key Features

• Multiscale Processing: MViT processes both spatial and temporal information at multiple scales. It leverages multiple levels of spatial and temporal resolutions to capture features at different granularities, enabling the model to understand both global context and fine details within video sequences.

​

• Patch-based Processing: Similar to ViT, MViT divides video frames into fixed-size patches and flattens them into sequences, which are then fed into transformer layers.

​

• Self-Attention Mechanism: MViT employs self-attention mechanisms within its transformer layers to capture long-range dependencies and relationships between patches in both spatial and temporal dimensions.

​

​

​

​

​

MViT Model Performance

Precision�100 %

Recall�96%

First Place for most of phase 1

Accuracy�98%

Best Model

04

“The key is not the will to win... everybody has that. It is the will to prepare to win that is important.” —Bobby Knight

Performance On The Last Hidden Set

Precision�94 %

Recall�100%

F1-score�97%

Accuracy�97%

Our Performance : Lorethril

Happy Ending !

Thanks!

​

Do you have any

questions?

Mohammed

Alaa

Mohamed Ibrahim

Our Team