Test evolution in LTP
* All the information provided (in terms of responses) will be kept confidential.  

Definitions
* Variability in the source code base is expressed in terms of conditional compilation macro directives such as #ifdef, #ifndef, and #if.
*Test cases:  are organized in command files, i.e., a text file containing one test case per line. The first item on the line is the tag name of the test, by which pan will know it. Usually this should match the TCID or test case ID of the test program.
*Command files: are usually stored under the runtest directory of LTP and they are a convenient way of grouping test programs together to create custom test suites.
*Test programs: correspond to the source code of test cases. To make it easy to find test programs, they have been organized under a folder structure represented by testcases directory.
In Google anmelden, um den Fortschritt zu speichern. Weitere Informationen
1 - Select the description that best represents the role you played in the last five years? *
2 - Current company? *
If you are a free contributor let us know.
3 - How long have you been contributing to the LTP project? *
4 - In your view, what are the main challenges related to LTP test development? *
As shown by the graph below, the number of releases has been decreasing along the years. What do you know about it? Any opinion?
Bild ohne Titel
6 - In general, the LOC size of LTP test programs has increased significantly from the initial to the current release. However, the general rate of growth seems to be slowing down since 2011. In your perception, why this has occurred?
Lines of Code (LOC): our analysis did not consider comment and blank lines, i.e., we only counted lines that actually contain executable code. Please note that the 122 releases are represented in the figure, but the name of releases shown on the X-axis are a subset of the total.
Bild ohne Titel
7 - The average of Cyclomatic Complexity of test programs is showed in the Figure below. The Cyclomatic Complexity has been indicating that the LPT test programs are easy to maintain
McCabe’s Cyclomatic Complexity v(G) is a single quantity that estimates the control flow complexity of the code. For a single function, v(G) is equivalent to the number of conditional branches in the program, e.g. v(G) = 1 for a program consisting of only sequential statements, with no conditional branching in it. In the C language, the control flows include if-then-else, for, while, do, and case statements.
Bild ohne Titel
Auswahl löschen
8 - In the last years, the fraction of LTP test program functions with high v(G) is decreasing. In your opinion, why this occurred?
McCabe’s Cyclomatic Complexity v(G) is a single quantity that estimates the control flow complexity of the code. For a single function, v(G) is equivalent to the number of conditional branches in the program, e.g. v(G) = 1 for a program consisting of only sequential statements, with no conditional branching in it. In the C language, the control flows include if-then-else, for, while, do, and case statements.
9 - LTP presented a decreasing trend in MI values. Since this metric is used to analyze code quality, do you agree that code quality of LTP test programs is decreasing?
Maintainability Index (MI) is a single-number value for estimating the relative code maintainability. We used this metric to analyze the code quality, since it combines several other metrics that are generally agreed to be components of code quality: LOC, McCabe cyclomatic complexity and Halstead volume. This metric expresses the relative maintainability of a complete software project in a single number, which is a straightforward way to calculate and which would have good predictive value.
Bild ohne Titel
Auswahl löschen
10 - The percentage of comment lines has been decreasing. In your opinion, why this occurred?
Bild ohne Titel
11 - By processing the LTP test cases’ history, we extracted a dataset of changes linking them to their specific release, as can be seen in the figure below. Despite the size and complexity of LTP, adding and removing test cases occurs with a regular frequency
Please note that a logarithmic (log) scale is used in the vertical axis. This way, we emphasized similarities in trends among series values, since the number of test cases added and removed presented lower values compared to the ones kept
Bild ohne Titel
Auswahl löschen
12 - The figure below shows how long a distinct test case is kept in LTP. A large number of test cases remain in few releases (1-10), in your opinion, why this occurred?
Bild ohne Titel
13 - Analyzing the figure above, there are around of 1000 test cases that stay for more than 100 releases in LTP. In your opinion, what makes the permanence of these test cases so long?
14 - The Figure below shows the number of code files changes per directory. The kernel directory concentrates the major code file changes. In your opinion, why this occurred?
Here changes are considered any alteration in source code, e.g.,  if the code of one file in a release X changed in relation to next release X+1. Please note that a logarithmic (log) scale is used in the vertical axis. This way, we can better exhibit the values, since the number of code files changes in some directories presented lower values than others.
Bild ohne Titel
15 - The figure below shows the growth of LOF in LTP. We can see that as the LOF increases the LTP test programs becomes more complex.
Lines of Feature Code (LOF) is the number of lines of code that are linked to feature expressions. This metric indicates whether a small or a large fraction of the code base is variable. LOF values were extracted by counting the number of lines between #ifdefs and #endif in the test programs and summed up per release.
Bild ohne Titel
Auswahl löschen
16 - The amount of feature code represents high values (more than 20%) of the LTP code, in releases with small LOC size (1st, 2nd and 3rd releases), as can be seen in Figure below. However, there is a smaller percentage of variable source code in large releases (range 1-8%). The specification of configurable features (LOF code) is more complex in larger releases than in smaller ones.
Bild ohne Titel
Auswahl löschen
17 - Figure below shows the number of LOF over time per directories. The Kernel directory presented consistent growth of LOF for the entire period. In your opinion, why this occurred?
Bild ohne Titel
18 - The NOFC metric value is increasing over time, as can be seen in the figure below. The growth of NOFC has resulted in more complexity.
Number of Features Constants (NOFC): The NOFC metric reflects directly the configuration dimension and provides important insights regarding to variability and complexity. This metric shows the number of feature constants, i.e., the total number of macro names that are referred in, at least, one feature expression (#ifdef) in the source code and sum them per release.
Bild ohne Titel
Auswahl löschen
19 - The figure below shows the evolution of test programs with and without variability. We can observe that the number of test programs with variability present an evolution behavior similar to test programs without variability. Thus, variability is present in a representative part of LTP. Test programs that contains variability are more difficult to maintain *
Variability in the source code base is expressed in terms of conditional compilation macro directives such as #ifdef, #ifndef, and #if.
Bild ohne Titel
20 - The figure below shows the v(G) metric for normalized values, such as the average of the v(G) per function in test programs with and without variability. The complexity of functions in test programs with variability is similar to the functions in test programs without variability.
Bild ohne Titel
Auswahl löschen
21 - In LTP the feature scattering increases with the increasing use of lines of code and feature constants in feature expressions. This situation result in more complexity as the project grows
Scattering Degree (SD): The SD metric indicates the number of feature constants occurrences in different feature expressions.
Bild ohne Titel
Auswahl löschen
Senden
Alle Eingaben löschen
Gib niemals Passwörter über Google Formulare weiter.
Dieser Inhalt wurde nicht von Google erstellt und wird von Google auch nicht unterstützt. - Nutzungsbedingungen - Datenschutzerklärung

Sieht dieses Formular verdächtig aus? Bericht