1 of 19

Code Coverage

CS 240 – Advanced Programming Concepts

​

2 of 19

Code Coverage Overview

3 of 19

Code Coverage Overview

  • Measures of how much code is covered (exercised or invoked) by tests
  • Calculated by automated tools that instrument the code to measure coverage
  • Tools can be stand-alone or built-in to IDEs
  • Helps developers determine if they’ve written enough tests
  • Cannot determine if the tests actually test the code, or if the tests are good
  • There are different types (measures) of code coverage
  • Companies and/or teams often have code coverage standards (i.e. you must have at least 90% statement coverage)

​

4 of 19

Types of Code Coverage

  • Line coverage: lines executed by tests / total lines
  • Statement coverage: same as line coverage if every statement is on it’s own line
  • Branch coverage: statement coverage + a measure of how many unique code branches were executed
    • In an if statement with no else, do tests exercise the cases where the if statement is both true and false?
    • Statement coverage wouldn’t detect whether the false case is covered
  • Function coverage: which functions are covered by tests
    • Useful to identify untested functions even if you have high statement or branch coverage
  • There are other measures that are less used and less important

5 of 19

Thoughts On What is “good” Coverage

  • There is no universally accepted or “right” answer
  • IDE generated code (such as getters and setters) doesn’t need to be covered with tests and should be excluded from measurements
  • Standards approaches:
    • 100% branch coverage of non-IDE generated code
      • It’s hard to cover UI code so exceptions are often made
      • It’s also unnecessary and wasteful to to cover simple, IDE generated code, like getters and setters
    • Some percentage (i.e. 90%) to account for code that shouldn’t have tests
      • Any percentage is arbitrary and may not be appropriate for all code written by a team or organization

6 of 19

7 of 19

Code Coverage Tools

8 of 19

Java Code Coverage Tools

  • Proprietary / built-in to IDEs like Intellij
  • Cobertura: Open-source, command line
  • JaCoCo: Open-source, command line but has Intellij plugin
  • Parasoft JTest: Nice but expensive
  • Many others

​

  • Command line tools can be integrated into an automated build / deployment process
  • Built-in or IDE plugin tools are useful for individual developers

9 of 19

10 of 19

Intellij Code Coverage

11 of 19

Using Intellij’s Built-in Code Coverage Tool

  • Right-click a test or test directory
    • Select: More Run/Debug -> Run '<test identifier>' with Coverage
  • Summary results appear in a coverage pane
  • Detailed results are shown with green and red line markings in open editor panes
  • Provides line coverage (not statement coverage)
  • Branch coverage is not enabled by default

12 of 19

Running Tests with Coverage

13 of 19

Coverage Results

14 of 19

15 of 19

Enabling Branch Coverage in Intellij

16 of 19

Enable Branch Coverage

  • Select Edit Configurations…
    • Open the dropdown at the top of the Intellij window (left of the green triangle “run” button)
  • Select Modify Options
  • Under "Code Coverage", select "Enable Branch Coverage and Test Tracking"
  • Click Apply and/or OK
  • Run the tests using the configuration you just modified:
    • If the run configuration was for a single test, run that test with coverage
    • If the run configuration was for a test directory, run that directory with coverage
    • Can also select the edit configuration in the dropdown and press the “Run with Coverage (shield) Icon”

17 of 19

Enable Branch Coverage (cont.)

18 of 19

Branch Coverage Result

The lines marked in yellow are partially covered branches.

19 of 19