1 of 68

PreAxC: Error Distribution Prediction for

Approximate Computing Quality Control using

Graph Neural Networks

Lakshmi Sathidevi, Abhinav Sharma, Nan Wu, Xun Jiao, Callie (Cong) Hao

Lakshmi S

Graduate Researcher,

Sharc Lab,

Georgia Tech

The 24th International Symposium on Quality Electronic Design (ISQED'23)

+

x

+

+

x

+

1

2

3

P(δ)

δ

2 of 68

Objective

  • Background
    • AxC trades-off accuracy for energy savings
    • Optimal trade-off has to be obtained for maximum energy savings

​

2

Data & Representation

Task & Models

Objective

Experiments

Results

3 of 68

Objective

  • Challenges
    • Existing works use simple error statistics
    • Lack of accurate & informative error models for AxC

​

  • Background
    • AxC trades-off accuracy for energy savings
    • Optimal trade-off has to be obtained for maximum energy savings

​

3

Data & Representation

Task & Models

Objective

Experiments

Results

4 of 68

Objective

  • Challenges
    • Existing works use simple error statistics
    • Lack of accurate & informative error models for AxC

​

  • Objective
    • Fast, accurate and informative error modelling
    • Error distribution prediction for approximate computation graphs
    • Employ GNNs for expressive and efficient learning

​

  • Background
    • AxC trades-off accuracy for energy savings
    • Optimal trade-off has to be obtained for maximum energy savings

​

4

Data & Representation

Task & Models

Objective

Experiments

Results

5 of 68

Application: Quality Control

5

Data & Representation

Task & Models

Objective

Experiments

Results

6 of 68

Application: Quality Control

6

Data & Representation

Task & Models

Objective

Experiments

Results

7 of 68

Application: Quality Control

7

Data & Representation

Task & Models

Objective

Experiments

Results

8 of 68

Application: Quality Control

8

Data & Representation

Task & Models

Objective

Experiments

Results

PreAxC

9 of 68

Application: Quality Control

9

Data & Representation

Task & Models

Objective

Experiments

Results

PreAxC

10 of 68

Application: Quality Control

10

Data & Representation

Task & Models

Objective

Experiments

Results

PreAxC

11 of 68

11

Data & Representation

Task & Models

Objective

Experiments

Results

PreAxC

12 of 68

12

Data & Representation

Task & Models

Objective

Experiments

Results

13 of 68

Data

13

Data & Representation

Task & Models

Objective

Experiments

Results

14 of 68

Data

14

Data & Representation

Task & Models

Objective

Experiments

Results

15 of 68

Data

15

Data & Representation

Task & Models

Objective

Experiments

Results

16 of 68

Data

16

Data & Representation

Task & Models

Objective

Experiments

Results

17 of 68

Data

17

Data & Representation

Task & Models

Objective

Experiments

Results

18 of 68

Data

18

Data & Representation

Task & Models

Objective

Experiments

Results

19 of 68

Graph Representation

19

Data & Representation

Task & Models

Objective

Experiments

Results

20 of 68

Graph Representation

20

Data & Representation

Task & Models

Objective

Experiments

Results

+

+

x

21 of 68

Graph Representation

21

Data & Representation

Task & Models

Objective

Experiments

Results

+

+

x

22 of 68

Graph Representation

22

Data & Representation

Task & Models

Objective

Experiments

Results

+

+

x

[0, 1, 0.2, 0.1]

[0, 0, 0, 0]

[1, 0, 0, 0]

23 of 68

Graph Representation

23

Data & Representation

Task & Models

Objective

Experiments

Results

+

+

x

[0, 1, 0.2, 0.1]

[0, 0, 0, 0]

[1, 0, 0, 0]

24 of 68

Graph Representation

24

Data & Representation

Task & Models

Objective

Experiments

Results

+

+

x

[0, 1, 0.2, 0.1]

[0, 0, 0, 0]

[1, 0, 0, 0]

GNN model used: Graph Isomorphism Network (GIN)[1]

[1] Keyulu Xu, Weihua Hu, Jure Leskovec, & Stefanie Jegelka. (2019). How Powerful are Graph Neural Networks?

25 of 68

Task

25

Data & Representation

Task & Models

Objective

Experiments

Results

26 of 68

Task

26

Data & Representation

Task & Models

Objective

Experiments

Results

+

x

+

+

x

+

1

2

3

O1

27 of 68

Task

27

Data & Representation

Task & Models

Objective

Experiments

Results

+

x

+

+

x

+

1

2

3

O1

P(δ)

δ(O1)

28 of 68

Approach

28

Data & Representation

Task & Models

Objective

Experiments

Results

29 of 68

Approach

29

Data & Representation

Task & Models

Objective

Experiments

Results

30 of 68

Approach

30

Data & Representation

Task & Models

Objective

Experiments

Results

31 of 68

Contrasting with Existing Methods

31

Data & Representation

Task & Models

Objective

Experiments

Results

32 of 68

Contrasting with Existing Methods

32

Data & Representation

Task & Models

Objective

Experiments

Results

33 of 68

Contrasting with Existing Methods

33

Data & Representation

Task & Models

Objective

Experiments

Results

34 of 68

Graph Expansion

34

Data & Representation

Task & Models

Objective

Experiments

Results

35 of 68

Graph Expansion

35

Data & Representation

Task & Models

Objective

Experiments

Results

+

x

+

+

x

+

36 of 68

Graph Expansion

36

Data & Representation

Task & Models

Objective

Experiments

Results

+

x

+

+

x

+

+

x

+

+

x

+

37 of 68

Graph Expansion

37

​

Shortest Dist.

​

No. of hops b/w A&B w/o new edge

Data & Representation

Task & Models

Objective

Experiments

Results

+

x

+

+

x

+

+

x

+

+

x

+

Edge Features

0

1

New/Existing

38 of 68

Graph Expansion Ablation Study

38

Data & Representation

Task & Models

Objective

Experiments

Results

39 of 68

Graph Expansion Ablation Study

39

Data & Representation

Task & Models

Objective

Experiments

Results

40 of 68

Graph Expansion Ablation Study

40

Data & Representation

Task & Models

Objective

Experiments

Results

41 of 68

Graph Expansion Ablation Study

41

Data & Representation

Task & Models

Objective

Experiments

Results

42 of 68

Graph Expansion Ablation Study

42

Data & Representation

Task & Models

Objective

Experiments

Results

43 of 68

Graph Expansion Ablation Study

43

Data & Representation

Task & Models

Objective

Experiments

Results

44 of 68

Performance

44

Data & Representation

Task & Models

Objective

Experiments

Results

45 of 68

Performance

45

Data & Representation

Task & Models

Objective

Experiments

Results

DFG Trav: Manual Traversal of Computation Graph

BN: Bayesian Network

REM: Relative Error Mean

KL: Kullback–Leibler divergence

BD: Bhattacharyya Distance

[2] Chaofan Li et al. Joint precision optimization and high level synthesis for approximate computing. In DAC, pages 1–6. IEEE/ACM, 2015.

[3] Marcello Traiola et al. Probabilistic estimation of the application-level impact of precision scaling in approximate computing applications.Microelectronics Reliability, 102:113309, 2019.

.

2

3

46 of 68

Performance

46

Data & Representation

Task & Models

Objective

Experiments

Results

DFG Trav: Manual Traversal of Computation Graph

BN: Bayesian Network

REM: Relative Error Mean

KL: Kullback–Leibler divergence

BD: Bhattacharyya Distance

2

3

47 of 68

Performance

47

Data & Representation

Task & Models

Objective

Experiments

Results

DFG Trav: Manual Traversal of Computation Graph

BN: Bayesian Network

REM: Relative Error Mean

KL: Kullback–Leibler divergence

BD: Bhattacharyya Distance

2

3

48 of 68

Performance

48

Data & Representation

Task & Models

Objective

Experiments

Results

DFG Trav: Manual Traversal of Computation Graph

BN: Bayesian Network

REM: Relative Error Mean

KL: Kullback–Leibler divergence

BD: Bhattacharyya Distance

2

3

49 of 68

Performance

49

Data & Representation

Task & Models

Objective

Experiments

Results

DFG Trav: Manual Traversal of Computation Graph

BN: Bayesian Network

REM: Relative Error Mean

KL: Kullback–Leibler divergence

BD: Bhattacharyya Distance

2

3

50 of 68

Performance

50

Data & Representation

Task & Models

Objective

Experiments

Results

DFG Trav: Manual Traversal of Computation Graph

BN: Bayesian Network

REM: Relative Error Mean

KL: Kullback–Leibler divergence

BD: Bhattacharyya Distance

2

3

Lesser is Better

51 of 68

Performance

51

Data & Representation

Task & Models

Objective

Experiments

Results

DFG Trav: Manual Traversal of Computation Graph

BN: Bayesian Network

REM: Relative Error Mean

KL: Kullback–Leibler divergence

BD: Bhattacharyya Distance

2

3

Lesser is Better

52 of 68

Performance

52

Data & Representation

Task & Models

Objective

Experiments

Results

53 of 68

Performance

53

Data & Representation

Task & Models

Objective

Experiments

Results

54 of 68

Performance

54

Data & Representation

Task & Models

Objective

Experiments

Results

55 of 68

Performance

55

Data & Representation

Task & Models

Objective

Experiments

Results

56 of 68

Performance

56

Data & Representation

Task & Models

Objective

Experiments

Results

57 of 68

Performance

57

Data & Representation

Task & Models

Objective

Experiments

Results

58 of 68

Key Results

  • Model-free approach performs well w.r.t performance metrics

​

58

Data & Representation

Task & Models

Objective

Experiments

Results

59 of 68

Key Results

  • Model-free approach performs well w.r.t performance metrics
  • Model-based approach learns characteristic nuances

​

59

Data & Representation

Task & Models

Objective

Experiments

Results

60 of 68

Key Results

  • Model-free approach performs well w.r.t performance metrics
  • Model-based approach learns characteristic nuances
  • No re-training required for unseen graphs
  • Can predict for real DFGs when only synthetic DFGs seen during training.

60

Data & Representation

Task & Models

Objective

Experiments

Results

61 of 68

Conclusion

  • We proposed PreAxC

​

61

62 of 68

Conclusion

  • We proposed PreAxC
  • Features:
    • Accurate
    • Informative with input-awareness
    • Generalization capability

​

62

63 of 68

Conclusion

  • We proposed PreAxC
  • Features:
    • Accurate
    • Informative with input-awareness
    • Generalization capability
  • We proposed two approaches:
    • Model-free using histogram
    • Model-based using GMM

​

63

64 of 68

Conclusion

  • We proposed PreAxC
  • Features:
    • Accurate
    • Informative with input-awareness
    • Generalization capability
  • We proposed two approaches:
    • Model-free using histogram
    • Model-based using GMM
  • We also proposed graph expansion

​

64

65 of 68

Conclusion

  • We proposed PreAxC
  • Features:
    • Accurate
    • Informative with input-awareness
    • Generalization capability
  • We proposed two approaches:
    • Model-free using histogram
    • Model-based using GMM
  • We also proposed graph expansion
  • Works for:
    • Transductive setting
    • Inductive setting

65

66 of 68

Conclusion

  • Future work:
    • Testing on more real-world benchmarks
    • Discussing how output error dist. affects the final application.

​

​

66

67 of 68

Acknowledgement

  • I would like to acknowledge Dr. Callie Hao - Sharc Lab @ Georgia Tech.
  • Partially supported by NSF under Grant No.2202329.

​

67

68 of 68

If you have any questions…

Please contact:

  1. Callie Hao: callie.hao@gatech.edu
  2. Lakshmi S: lsathidevi3@gatech.edu

​

​

68