1 of 2

LLFPTrax: Tracking ill-conditioned floating-point

inputs using relative error amplification in LLVM

​

SOME UNSTABLE NUMERICAL PROBLEMS/APPLICATIONS

Tanmay Tirpankar1, Ignacio Laguna2, Ganesh Gopalakrishnan1

1University of Utah, 2Lawrence Livermore National Laboratory

REFERENCES

[1] David Goldberg. 1991. What every computer scientist should know about floating-point arithmetic. ACM Comput. Surv. 23, 1 (March 1991), 5–48. https://doi.org/10.1145/103162.103163

[2] http://people.eecs.berkeley.edu/~demmel/cs267/lecture21/lecture21.html.

[3] https://en.wikipedia.org/wiki/Lorenz_system, 2020.

[4] Daming Zou, Muhan Zeng, Yingfei Xiong, Zhoulai Fu, Lu Zhang, and Zhendong Su. 2019. Detecting floating-point errors via atomic conditions. Proc. ACM Program. Lang. 4, POPL, Article 60 (January 2020), 27 pages. https://doi.org/10.1145/3371128

5

LLFPTrax OVERVIEW

FUTURE PLANS

  • Making the tool more scalable
  • Support offline/partial processing
  • Support more operators
  • Additional tracing/analysis facilities
  • Automated repair

6

FLOATING-POINT ERROR ANALYSIS and ROLE OF LLVM

Growing need for floating-point error analysis :

  • Practitioners have inadequate analysis help from Floating-Point rounding [1]
  • Error analysis targeting multiple objectives
  • Programs suffer Instabilities.

Motivations for LLVM to support Error Analysis:

  • Helps close the current gap between languages in actual use and today's rigorous floating-point analysis methods.
  • Enables correctness analysis of today's HPC programs.
  • Rigorous mixed-precision optimization support in LLVM.

1

2

  • f(x) = x / (1-x)

​

​

x

0.93

0.94

-0.9300

-0.9400

y

13.28

15.66

-0.4818

-0.4845

  • f(x) = log(1-x)/x [2]

​

  • Lorenz System of Equations [3]

​

​

True plot of f(x)

Plot of f(x) using IEEE floating-point

Plot of f(x) using IEEE floating-point near x = 0

Plot of f(x) computed carefully for IEEE floating-point

Correct Program:

3

BACKGROUND

If f is differentiable

Per Operator Amplification [4]

Node

Amplification Factor

N1

1.00e-14

N2

2.00e-00

N3

9.99e-01

N4

2.00e-00

4

METHODOLOGY OVERVIEW

  • Computing Amplification
  • Handling Branch Instructions

Template ID: basicprofessional Size: 36x48

2 of 2

LLFPTrax: LLVM-Based Tracking of Ill-Conditioned Floating-Point

inputs Based on Relative-Error Amplification

SOME UNSTABLE NUMERICAL EXPRESSIONS/PROGRAMS

Tanmay Tirpankar1, Ignacio Laguna2, Ganesh Gopalakrishnan1

1University of Utah, 2Lawrence Livermore National Laboratory

REFERENCES

[1] David Goldberg. 1991. What every computer scientist should know about floating-point arithmetic.

[2] http://people.eecs.berkeley.edu/~demmel/cs267/lecture21/

lecture21.html.

[3] https://en.wikipedia.org/wiki/Lorenz_system, 2020.

[4] Daming Zou, Muhan Zeng, Yingfei Xiong, Zhoulai Fu, Lu Zhang, and Zhendong Su. 2019. Detecting floating-point errors via atomic conditions.

This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344 (LLNL-CONF-832228), DOE ASCR Award Number DE-SC0022252 and NSF CISE Awards 2124100 and 1956106

5

LLFPTrax OVERVIEW

FUTURE PLANS

  • Scalable handling of production HPC programs and applications
  • Support for user-guided selective analysis to handle scale
  • Additional tracing/analysis facilities to pinpoint computation phases causing instability
  • Automated repair informed by Amplification Factors

6

FLOATING-POINT ERROR ANALYSIS and ROLE OF LLVM

Growing need for floating-point error analysis :

  • HPC and ML programs suffer from difficult-to-diagnose errors due to floating-point rounding [1]
  • Flexible tooling needed to track down multiple types of numerical errors
  • We focus on instability caused by relative error

Why support FP Error Analysis in LLVM?

  • Most current methodologies for rigorous error analysis employ their own ad hoc input languages
  • Long-term: mixed-precision support in LLVM

1

2

  • f(x) = x / (1-x)

​

​

x

0.93

0.94

-0.9300

-0.9400

y

13.28

15.66

-0.4818

-0.4845

  • f(x) = log(1-x)/x [2]

​

  • Lorenz System of Equations [3]
    • System eventually diverges due to numerical instability.

​

​

True plot of f(x)

Plot of f(x) using IEEE floating-point

Plot of f(x) using IEEE floating-point near x = 0

Plot of f(x) computed carefully for IEEE floating-point

Correct Program:

3

BACKGROUND

If f is differentiable

Per Operator Amplification [4]

Node

Amplification Factor

N1

1.00e-14

N2

2.00e-00

N3

9.99e-01

N4

2.00e-00

4

METHODOLOGY OVERVIEW

  • Computing Amplification
  • Handling Branch Instructions

x10-15

  • Amplification factors computed in the forward pass
  • Amplification paths ranked

Template ID: basicprofessional Size: 36x48