1 of 27

5th WORKSHOP

on Road Traffic Monitoring

Team 3: Marc Gorriz, Guillermo Torres, Cristina Maldonado and Ivan Caminal

Video Surveillance for Road Traffic Monitoring

2 of 27

Outline Cristina - Team 3

2

Background Estimation
Region Tracking
Speed Computation
Application
Conclusions & Future Work

3 of 27

Background Estimation Pipeline Cristina- Team 3

3

Mask Post Processing

Morphological Operators

Hole Filling
Area Filtering

Shadow Removal

Opencv → MOG2^b

Video Preprocessing

Background Estimation

Obtain binary masks to differentiate foreground and background on a scene in order to detect moving objects on a video.

Background Subtraction

Gaussian Modelling
Adaptive Modelling
Opencv → MOG^a

Video Stabilization

Block Matching
Web Application^c

a,b: MOG, MOG2 opencv functions

c: video stabilize

GOAL

4 of 27

Background Estimation Pipeline 0- Team 3

4

Mask Post Processing

Morphological Operators

Hole Filling
Area Filtering

Shadow Removal

Opencv → MOG2^b

Video Preprocessing

Background Estimation

Obtain binary masks to differentiate foreground and background on an image in order to detect moving objects on a video.

Background Subtraction

Gaussian Modelling
Adaptive Modelling
Opencv → MOG^a

Video Stabilization

Block Matching
Web Application^c

a,b: MOG, MOG2 opencv functions

c: video stabilize

GOAL

	Highway	Fall	Traffic
1-Hole Filling	4 connectivity	4 connectivity	4 connectivity
2-Opening	s.e. 4x4 (square)	s.e. 16x16 (square)	s.e. 5x5 (square)

Best post processing

5 of 27

Background estimation Ivan - Team 3

5

Best predicted masks*

Estimated μ

ρ = 0.59 (optimal)

*Adaptive modeling + best post processing

Highway sequence [1050 - 1350]

ρ = 0.59

ρ = 0.01

ρ = 0.1

ρ = 0.3

Training μ w.r.t. ρ

6 of 27

Background estimation Ivan - Team 3

6

Idea: Set ρ to [0.01, 0.1] and optimize α

Problem: Our implementation of the Gaussian adaptive model was wrong.

Best

f1-score 0.09

Implications...

We finally solve it in this commit.

We get inspiration in how to resolve it by looking this team (of previous year workshop).

7 of 27

Background estimation Ivan - Team 3

7

Before

After

Comparison

Background Estimation

Adaptive Modelling

Implications:

8 of 27

Background estimation Ivan - Team 3

8

Before

After

	Highway	Traffic
1-Opening	s.e. 4x4 (square)	s.e. 11x11 (square)
2-Dilation	x4 iter*	x8 iter*
3-Hole Filling	4 conect.	4 conect.
4-Erosion	x4 iter*	x8 iter*

Background Estimation

Mask Post Processing

Adaptive Modelling

Morphological Operators

Implications:

*Iterations with 3x3 s.e.

Best post processing (After correction)

9 of 27

Background estimation Ivan - Team 3

9

Before

After

Comparison

Background Estimation

Mask Post Processing

Adaptive Modelling

Morphological Operators

Implications:

10 of 27

Background estimation Ivan - Team 3

10

Updated results!

*

Modified slide from week 2 feedback

*

11 of 27

Background estimation Ivan - Team 3

11

Before

After

Highway [1050 - 1350]

Traffic [950 - 1050]

12 of 27

Object Tracking Marc - Team 3

12

Original

Mask

Connected Components

New Track

Track #2

id: 2

centroid: X age: 0

area: X visible: True

bbox: X track_filter: kalman

speed: 0 ....

Area Filtering

13 of 27

Object Tracking Marc - Team 3

13

Track update

For each visible track we need to update its new centroid position.

After the update, we can consider the following cases:

Create new tracks
Make tracks invisible
Delete old invisible tracks

At each frame . . .

14 of 27

Object Tracking Marc - Team 3

14

Track update

Track #2

id: 2

centroid: X age: 0

area: X visible: True

bbox: X track_filter: kalman

speed: 0 ....

Considering only Track #2

15 of 27

Object Tracking Marc - Team 3

15

Track update

The new centroid position is predicted using a Kalman filter (or others)

16 of 27

Object Tracking Marc - Team 3

16

Track update

The square distance is computed from the predicted centroid to the other tracks.

The prediction is assigned to the nearest track.

17 of 27

Object Tracking Marc - Team 3

17

Track #2

id: 2

centroid: age: 0

area: X visible: True

bbox: X track_filter: kalman

speed: 0 ....

Track update

We update the track centroid

18 of 27

Object Tracking Marc - Team 3

18

KALMAN FILTER

MEDIAN FLOW

KCF

BOOSTING

H I G H W A Y

19 of 27

Object Tracking Marc - Team 3

19

KALMAN FILTER

MEDIAN FLOW

KCF

BOOSTING

H I G H W A Y

20 of 27

Object Tracking Marc - Team 3

20

KALMAN FILTER

MEDIAN FLOW

KCF

BOOSTING

H I G H W A Y

21 of 27

Object Tracking Marc - Team 3

21

KALMAN FILTER

MEDIAN FLOW

KCF

BOOSTING

H I G H W A Y

22 of 27

Speed Estimation Guillermo - Team 3

22

Projective transformation to obtain bird's-eye perspective, estimating the fundamental matrix using the 8-point algorithm*.

Applying the fundamental matrix, we can transform only the positions of the centroid to save computational resources.

Homography estimation

8-point algorithm

*H.C. Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293:133–135, Sept 1981.

Code by Peter Kovesi, School of Computer Science & Software Engineering UWA, Australia

Slide credit: team1, class2017

23 of 27

Speed Estimation Guillermo - Team 3

23

Homography estimation

8-point algorithm

Speed computation

Simple Assumption

c1

c2

d (px)

D’ (m)

D (px)

Simple assumption

* updating speed every 8 frames

24 of 27

Applications Guillermo - Team 3

24

TRAFFIC

Road traffic monitoring

Density on the road

1 or 2 vehicles → Low Traffic
3 or 4 vehicles → Moderate Traffic
5 or more vehicles → High Traffic

Speed monitoring
Trajectory estimation
Imprudent driving detection

HIGHWAY

CUSTOM

25 of 27

Applications Guillermo - Team 3

25

TRAFFIC

Road traffic monitoring

Density on the road

1 or 2 vehicles → Low Traffic
3 or 4 vehicles → Moderate Traffic
5 or more vehicles → High Traffic

Speed monitoring
Trajectory estimation
Imprudent driving detection

HIGHWAY

CUSTOM

26 of 27

Conclusions & Future Work

26

We learnt that errors in the first stages of the pipeline can produce bad results and the optimal subsequent stages can not be reutilized.

We have appreciated the important role played by having good predicted masks to implement object tracking.

We developed a traffic monitoring system modularized by a pipeline to track cars, count them, estimate their trajectory and speed.

As further work we can improve the overall system by segmenting better the foreground objects.

Tracking could be improved using more sophisticated filters.

We are happy with the work done and with all the learning process.

27 of 27

Guillermo Torres

Ivan Caminal

Marc Gorriz

Cristina Maldonado

Github Repository