1 of 16

Searching the Deployable Convolution Neural Networks for GPUs

October 07, 2022

Presenter – Joon Sung Park

Wang, Linnan, et al. "GPUNet: Searching the Deployable Convolution Neural Networks for GPUs." CVPR (2022).

2 of 16

CONTENTS

2

  1. Motivation
  2. Overview
  3. Method
  4. Experiment
  5. Conclusion

3 of 16

Motivation

4

Research phase

Deployment phase

Development process

4 of 16

Motivation

4

Research phase

Deployment phase

Development process

5 of 16

Motivation

4

  • NAS (Neural Architecture Search)
    • Search space
    • Evaluation strategy
    • Search strategy

​

​

​

6 of 16

Overview

4

7 of 16

Method

4

8 of 16

Method - hyperparameters

​

4

  • Hyperparameters
    • Lower discrepency
    • Faster convergence
    • Stable estimation
  • Network
    • Convolution layers
    • Inverted residual blocks (IRBs)
    • Fused IRBs

​

​

​

​

9 of 16

Method – sobol sampling

4

  • Process
    • Rather generating random numbers, it generates a uniform distribution in probability space

​

  • Pro
    • Lower discrepancy
    • Faster convergence
    • Stable estimation

​

10 of 16

Method - TensorRT

4

  • Measurement tool
  • Methods
    • Quantization
    • Graph optimization
    • Kernel auto tuning
    • Multi-stream execution

​

​

11 of 16

Method – LA-MCTS black box optimization

4

  • LA-MCTS boosted Bayesian optimization

12 of 16

Method

4

13 of 16

Experiment

4

  • Dataset
    • ImageNet
  • Training procedure
    • Pretraining – 300 epochs
    • Finetuning – 150 epochs

​

  • Machines
    • DGX A100 with 8x A100 80 GB

14 of 16

Experiment

4

15 of 16

Conclusion

4

  • It is able to customize neural models with industrial tool (e.g. tensorRT)
  • It works in various tasks & search space.
  • It is directly accessible.

​

16 of 16

THANK YOU