Uncertainty Aware Proposal Segmentation for Unknown Object Detection

WACV DNOW Workshop

​

Yimeng Li and Jana Kosecka

Task: Anomaly Detection in Driving Scenes

• In autonomous driving, unknown objects can appear on the road and become a potential threat to safety.

​

Task: Anomaly Detection in Driving Scenes

• Detect these anomalies not encountered by an object detector/semantic segmentation model in the training data.

​

Existing Methods

[Detecting the unexpected via image resynthesis,

Krzyzstof et al. ICCV’2019]

[Pixel-wise anomaly detection in complex driving scenes,

Giancarlo et al. CVPR’2021]

Approach: Proposal Segmentation and Classification

• We do semantic segmentation on object proposals accompanied by uncertainty estimation through RBF-Network. Pixels with high uncertainty to classify into known background categories are masked out as object candidates.
• Further classification between known object categories on the masked-out pixels determines if it is an out-of-dist object.

​

Proposal Segmentation

• The proposal segmentation model takes
• An object proposal generated by RPN [1] or EdgeBox [2]
• An associated feature map from MaskRCNN [3] or DeeplabV3+ [4]
• Label each pixel of the proposal into one of the known semantic categories and estimate its uncertainty
• Pixels with high uncertainty on the background classes forms the object mask

[1] Ren et al. 2017, [2] Zitnick et al. 2014, [3] He et al. 2017, [4] Chen et al. 2018

Proposal Classification

• Apply max-pooling on the feature map associated with the mask
• Pass the resulting feature to Radial Basis Function Network (RBFN) for classification with uncertainty

Radial Basis Function Network (RBFN)

[1] Amersfoort et al. 2020

Improve RBFN: Boundary Regularization

• RBFN is susceptible to feature collapse if having multiple conv layers [1]
• Conflict with batch normalization and gradient penalty
• Boundary Regularization
• enforce a uniform distribution for the boundary pixels and maximize the classification performance on the in-distribution pixels

​

[1] Jeremiah et al. 2020

Toy Example in 2D Dimension

• Use two Gaussian blobs (red and green) to represent known semantic features
• Uniform distributed blue points represent unknown semantic features in (a)
• Point brightness represents estimated certainty if the points are classified into the center red gaussian blob in (b) (c) (d) (e)

Experiment Setup: Proposal Segmentation and Whole Image Segmentation

• Training: Cityscapes [1]
• Evaluation: Lost&Found [2], Fishyscapes [3], RoadAnomaly [4]
• Objective: Detect the anomalous/out-of-dist object pixels
• Metrics:
• AUROC (AC): Area under the ROC curve
• Average Precision (AP): average over in-distribution and out-of-distribution
• FPR95: False positive rate at 95% true positive rate

[1] Marius et al. 2017, [2] Pinggera et al. 2016, [3] Blum et al. 2019, [4] Lis et al. 2019

Results on Proposal Segmentation

[1] Amersfoort et al. 2020, [2] Krzysztof et al. 2019

Results on Whole Image Segmentation

[1] Amersfoort et al. 2020, [2] Krzysztof et al. 2019, [3] Giancarlo. 2021

Proposal Segmentation

Whole Image Segmentation

Detect Mask-RCNN False Predictions

Result on Indoor Scenes

• Trained on ADE20K [1]
• Evaluated on AVD [2]

[1] Zhou et al. 2017, [2] Ammirato et al. 2017

Conclusion

• Semantic Segmentation on detected Object Proposals reduces the distraction from the background.

​

• Using Radial Basis Function Network and Boundary Constraint is well suited for uncertainty quantization.

​

• Competitive performance on multiple datasets and indoor scenes.

​

• Thank you