1 of 20

An Executable Security Benchmark Suite for Server-Side JavaScript

Masudul Hasan Masud Bhuiyan, Adithya Srinivas Parthasarathy,

Nikos Vasilakis, Michael Pradel, Cristian-Alexandru Staicu

Masudul Hasan Masud Bhuiyan| May 2023 | ICSE 2023

2 of 20

JavaScript: Everything, Everywhere

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

2

https://github.com/AhmedZerouali/vulnerability_analysis

http://www.modulecounts.com/

3 of 20

Security in JavaScript

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

3

NDSS’18

RAID’20

USENIX’18

ESEC/FSE’18

ESEC/FSE’21

USENIX’23

NDSS’21

Slows down research, wastes time and effort.

No way to compare different methods.

No security-oriented benchmark suite for Node.js

4 of 20

Properties of an Ideal Benchmark

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

4

Realistic

Benchmark suite should include diverse real-world applications for representativeness and generalizability.

Executable

Executable exploits should be included for runtime analysis and detection/mitigation evaluation.

Two-sided

Including both vulnerable and fixed versions enables measuring the false positive rate of different detection and mitigation techniques.

Vetted

Manually validating the benchmark suite to confirm vulnerabilities, type of security problem, and metadata.

5 of 20

Methodology

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

5

Source

Snyk
GitHub Advisories
Huntr.dev

Security Advisories

6 of 20

Methodology

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

6

Source

Focus on vulnerabilities.

Does not include malicious packages.

Include only when there is an executable exploit.
Exclude vulnerabilities that

Cannot be installed
Cannot be reproduced
Incompatible with the setup (e.g., operating system)

Filtering

Candidate Vulnerabilities

7 of 20

Methodology

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

7

source

Filtering

Categories

ReDoS

Prototype pollution

Code injection

Command injection

Path traversal

192

98

40

101

169

Type of vulnerability

Nb. exploits

8 of 20

Methodology

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

8

Filtering

Categories

PoC

If the public vulnerability report contains a proof-of-concept exploit, we adapt it and include in the suite.
Otherwise, create the PoC ourselves utilizing the natural language description of the exploit.

Creating Proof of Concept (PoC)

Filtering

source

9 of 20

Methodology

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

9

Implementation

Exploit for lodash’s vulnerability

Filtering

Categories

PoC

Filtering

source

Jest

10 of 20

Methodology

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

10

Implementation

Exploit for lodash’s vulnerability

Security

Oracle

ReDos

Prototype pollution

Code injection

Command injection

Path traversal

Checks the existence of a variable called polluted in the global scope

Checks the target payload takes more than d=1 second to execute

Checks the existence of a custom-named file on the disk

Match the content served by the vulnerable package with flag.txt

Type of vulnerability

Oracle

To ensure the successful exploitation of each exploit, SecBench.js uses security oracle to observe that the exploit triggers an unforeseen, security-relevant action.

Filtering

Categories

PoC

Filtering

source

11 of 20

Methodology

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

11

Vulnerable version of the library

CVE identifier of the exploit if available

URLs of the public vulnerability report

Version number of the package where the exploit is patched

Link to the GitHub commit for the patch

Exact location in the code where the exploit happens

12 of 20

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

12

Realistic

Two-sided

Executable

Vetted

Includes 600 publicly available vulnerable npm packages.
Packages are included without any modification.

Each exploit: includes an executable proof of concept.
Each entry: includes an exploit oracle.

48% of the exploits in the suit include both vulnerable and fixed versions.

Each entry: analyzed, tested, and cross-checked by three researchers.

13 of 20

So What?

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

13

14 of 20

Example use case: Mislabeled Vulnerable Versions

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

14

Jest Test

Library

New Version

Vulnerable

Safe

15 of 20

Example use case: Mislabeled Vulnerable Versions

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

15

Zero-day vulnerability

Mislabeled versions

Safe

16 of 20

Example use case: Finding Flawed Fixes

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

16

Vulnerable

Fix

"__proto__.x"

"constructor.prototype.x"

Convict

How to find flawed fixes at a scale?

17 of 20

Example use case: Finding Flawed Fixes

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

17

Jest Test

Library

New Version

Vulnerable

Safe

Mutation

18 of 20

Example use case: Finding Flawed Fixes

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

18

"__proto__" → "constructor.prototype"

"__proto__" → ["__proto__"]

"__proto__": {...} → "constructor": {"prototype": {...}}

CVE-2021-23518, CVE-2021-23760, CVE-2021-23507 CVE-2021-23497, CVE-2021-23460, CVE-2021-23558 CVE-2022-25354, CVE-2022-25296, CVE-2022-25352

CVE-2021-23470

CVE-2022-22143, CVE-2022-24279

Mutation

Security advisories

19 of 20

Conclusion

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

19

@MHasan690

https://github.com/cristianstaicu/SecBench.js

20 of 20

JavaScript: Everything, Everywhere

Masudul Hasan Masud Bhuiyan - CISPA Helmholtz Center for Information Security

20

https://github.com/AhmedZerouali/vulnerability_analysis

http://www.modulecounts.com/