1 of 12

Automated Program Repair (2022-27)

MoE Tier 3

Abhik Roychoudhury

WN Chin, Ilya Sergey

8 Feb 2024

2 of 12

MoE Tier 3 APR – Launch with FSE2022

2

NUS APR Workshop � February 2024

3 of 12

MoE Tier 3: Automated Program Repair

Mission (2022-27): programming of tomorrow

“… provide foundations and tools for developing and maintaining trustworthy software systems, by detecting potential errors in them and by constructing correct fixes in a fully automated manner. As changes are introduced into programs in the course of their life cycle, the programs are being checked and “healed”, if needed, by applying automated repairs. Instead of treating a software system as a passive entity and validating it once and for all, we view it as a “living thing”, which can (provably) protect itself from pernicious changes, via APR.”

PRODUCTIVITY

EDUCATION

SECURITY

3

NUS APR Workshop � February 2024

4 of 12

MoE Tier 3: Automated Program Repair

4

NUS APR Workshop � February 2024

5 of 12

MoE Tier 3: Automated Program Repair�

Mission (2022-27)

Symbolic program analysis for trustworthy (evolving) software systems.
Test-suite guided program repair which generalizes to other tests.
Last mile repair for greater automation in code generation.�

Combine research from program analysis, verification, and test generation to achieve the the vision of self-healing software.

�Achievements (2022-23)

Most Influential Paper (10yr Test-of-time) at Intl Conf on Software Engineering (ICSE) 23 for ICSE13 paper “SemFix: Program Repair via Semantic Analysis”
Repair of Large Language Model -generated code [ICSE’2023]�🡪 Repairing auto-generate code.
(Mostly) Automated Proof Repair [PLDI’2023]�🡪 Maintaining proofs for evolving systems.
Intelligent tutoring system at NUS to support programming education.

PRODUCTIVITY

SECURITY

EDUCATION

5

NUS APR Workshop � February 2024

6 of 12

Static Analysis driven Program Repair (FSE23)

6

NUS APR Workshop � February 2024

7 of 12

Temporal Property guided Program Repair (FSE24)

Yahui Song¹, Xiang Gao², Wenhua Li¹, Wei-Ngan Chin¹, Abhik Roychoudhury¹

¹National University of Singapore, ²Beihang University

Compositional static analyzer via temporal properties.
Specified 17 APIs; found 515 vulnerabilities �from 1 million LOC; with a 90% fix rate.
Specification: a novel future-condition.
Specification inference via bi-abduction.
The inferred spec can be used to analysis

protocol applications, e.g., OpenSSL.

7

NUS APR Workshop � February 2024

8 of 12

Trustworthy Automated Programming (ICSE23)

Understanding the task description is still difficult

33% of bugs made by Codex are misaligned algorithms

Automated Repair of Programs from Large Language Models

Z Fan, X Gao, M Mirchev, A Roychoudhury, SH Tan

2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)

8

NUS APR Workshop � February 2024

9 of 12

Intelligent Tutoring System (ICSE24)

9

NUS APR Workshop � February 2024

10 of 12

First International APR Competition

https://apr-comp.github.io

10

NUS APR Workshop � February 2024

Student Assignments

02

Program Languages: Python, C
baseline-Python: Refactory (ASE’19)
baseline-C: ITSP (FSE’17)

Functionality Errors

01

Program Languages: C/C++ and Java
baseline-Java: Defects4J (ISSTA’14)
baseline-C/C++: ManyBugs (TSE’15)

AI Code Errors

03

Program Languages: Python, Java
baseline: LMDefects (ICSE’23)

11 of 12

11

NUS APR Workshop � February 2024

Competition Insights

1

Fix Localization is still a Challenge

Repair tools cannot identify fix locations that would lead to a plausible repair

2

Impact of Bit Rot is High

Dependencies used by the state of the art repair tools are outdated and does not work on newer software programs

3

Fine-Tuned LLM outperforms GPT4

Fine-tuned models for repair based on CodeLLAMA and StarCoder outperforms GPT4

4

Patch Validation is Overlooked

Test execution prioritization, test case selection and test case reduction can be used to improve patch validation

12 of 12

12

NUS APR Workshop � February 2024