1 of 4

WebNN Use Case Investigation:

Semantic Segmentation

Ningxin Hu, Intel

1/9/2019

2 of 4

Semantic Segmentation

  • WebNN semantic segmentation use case:
    • Pixel segmentation and replacement
    • Useful for web-based teleconference
    • Example models: DeepLabV3+ and MaskR-CNN
  • WebNN semantic segmentation example:
    • Use DeepLabV3+ MobileNet V2 model (mobilenetv2_coco_voc_trainval)
    • Segment the video stream from WebRTC and replace background with WebGL
    • Verified by both polyfill (WebGL and WASM) and WebNN POC with clDNN backend on Windows and Linux
    • Demo video

Segmentation

Background replacement

3 of 4

Requirements of DeepLabV3+ MobileNet V2 Model

Required Ops

NNAPI

MPS

ONNX

clDNN

Element-wise Add

ANEURALNETWORKS_ADD

MPSNNAdditionNode

Add

eltwise (mode as eltwise_mode::sum)

Average Pooling

ANEURALNETWORKS_AVERAGE_POOL_2D

MPSCNNPoolingAverageNode

AveragePool

pooling (mode as pooling_mode::average)

Concatenation

ANEURALNETWORKS_CONCATENATION

MPSNNConcatenationNode

Concat

concatenation

Convolution

ANEURALNETWORKS_CONV_2D

MPSCNNConvolutionNode

Conv

convolution

Depthwise Convolution

ANEURALNETWORKS_DEPTHWISE_CONV_2D

MPSCNNConvolutionNode (MPSCNNDepthWiseConvolutionDescriptor)

Conv with attributes["group"] equals to in_channels

convolution (weights split as depth_out)

Atrous Convolution

MPSCNNConvolutionNode (MPSCNNConvolutionDescriptor.dilationRate[XY])

Conv with attributes[“dilations”]

convolution (dilation)

Resize bilinear

ANEURALNETWORKS_RESIZE_BILINEAR (without support of “align_corners”)

MPSCNNUpsamplingBilinearNode

Upsample with attributes[“mode”] as bilinear (without support of “align_corners”)

By custom kernel (interp.cl)

Checkout details: WebNN POC supported models, supported ops and native mappings

4 of 4

Performance

  • CPU: Intel i7-8700 @3.20GHz, GPU: Intel UHD Graphics 630, OS: Ubuntu Linux 16.04.3
  • WebNN POC: Chromium 70.0.3503.0, clDNN: drop 8.0, Polyfill: WebGL backend is based on TensorFlow.js core 0.13.12, WASM backend is based on TensorFlow-Lite r1.12�