Unit Tests & Catch2
Advanced Computer Science
Introduction to Unit Testing
Unit testing is a software testing methodology where individual units or components of a software application are tested in isolation to ensure they function correctly.
Source: https://medium.com/@nairgirish100/my-personal-favorite-dilbert-strips-on-software-quality-be90b46e2f04
Benefits of Unit Testing
The benefits of unit testing include:
Popular unit testing frameworks, like JUnit for Java, NUnit for .NET, and Catch2 for C++, provide tools and conventions to streamline the creation and execution of unit tests.
Key aspects of unit testing
Anatomy of a Unit Test:
The anatomy of a unit test typically follows a structure that includes setup, execution, and assertions. Here's a breakdown of the key components or phases of a unit test:
Anatomy of a Unit Test:
Pseudo-Code | Unit Test Implementation (Catch2) |
// Test Fixture (Setup) initialize test data prepare the environment | // Test Fixture (Setup) TEST_CASE("Example Test Case") { MyClass obj; // Instantiate class to test |
// Act (Execution) result = codeUnderTest(parameters) | // Act (Execution) int result = obj.someFunction(2, 3); |
// Assert (Verification) assert(result == expectedValue, "Test failed: Unexpected result") | // Assert (Verification) REQUIRE(result == 5); |
// Teardown (Optional Cleanup) cleanup resources or revert changes | } |
Test-Driven Development (TDD)
Test-Driven Development (TDD) is a software development approach in which tests are written before the actual code is implemented. The TDD process follows a cycle known as the "Red-Green-Refactor" cycle, emphasizing short development iterations and continuous feedback. Here's an overview of the key principles and steps in Test-Driven Development:
Red-Green -Refactor Cycle | TDD Workflow |
Red | Write a Failing Test:
|
Green | Write the Minimum Code:
|
Refactor | Refactor:
|
| Repeat:
|
TDD Project Flow
TDD is an iterative and disciplined approach to software development that promotes code quality, maintainability, and rapid feedback. It is well-suited for building reliable and robust software systems.
Behavior Driven Development
Behaviour-driven development (BDD) is meant to be a more readable format by each stakeholder since it is in simple English. BDD is aligned with the Agile Development model. BDD is unlike Test-driven development (TDD) test cases which are written in programming languages like Java, Ruby, etc.,
· BDD explains the application behavior for the end-user while TDD focuses on how functionality is executed. Functionality Changes can be accommodated with fewer impacts on Behaviour-driven development as opposed to Test-driven development.
· Behaviour-driven development allows all the stakeholders to be on the same page with necessities which makes approval simple, contrasting to Test-driven development.
Behavior Driven Development
Behaviour-driven development (BDD) is meant to be a more readable format by each stakeholder since it is in simple English. BDD is aligned with the Agile Development model. BDD is unlike Test-driven development (TDD) test cases which are written in programming languages like Java, Ruby, etc.,
· BDD explains the application behavior for the end-user while TDD focuses on how functionality is executed. Functionality Changes can be accommodated with fewer impact on Behaviour-driven development as opposed to Test-driven development.
· Behaviour-driven development allows all the stakeholders to be on the same page with necessities which makes approval simple, contrasting to Test-driven development.
Behavior-Driven Development (BDD)
Behavior-Driven Development (BDD) is a software development approach that extends the principles of Test-Driven Development (TDD) by encouraging collaboration between developers, testers, and non-technical stakeholders. BDD emphasizes clear communication and shared understanding of the software's behavior through the use of natural language specifications. Here are the key aspects of the Behavior-Driven Development approach:
1. Collaboration:
BDD promotes collaboration between different roles in the development process:
2. Ubiquitous Language:
BDD encourages the use of a shared, domain-specific language that is accessible to both technical and non-technical team members. This language, often referred to as the "ubiquitous language," helps ensure that everyone involved in the project understands the requirements and goals.
3. Scenarios and Examples:
BDD scenarios are written in natural language and describe how the software should behave under certain conditions. Scenarios often take the form of "Given-When-Then" statements:
Behavior-Driven Development (BDD)
4. Tools and Frameworks:
BDD often involves the use of specific tools and frameworks designed for BDD, such as Cucumber, SpecFlow, Behave, or JBehave, depending on the programming language and platform.
5. Automated Tests:
Scenarios and examples written in natural language are translated into executable tests. These automated tests serve as both documentation and verification of the software's behavior. The tests are typically written using a BDD testing framework, such as Cucumber, SpecFlow, or Behave, which allows the translation of natural language specifications into executable code.
6. Continuous Feedback:
BDD provides continuous feedback throughout the development process:
7. Iterative Development:
BDD follows an iterative development cycle similar to TDD's Red-Green-Refactor cycle. The cycle involves writing a failing scenario, implementing the necessary code to make the scenario pass, and then refactoring as needed.
Behavior-Driven Development (BDD)
8. Feature Files:
BDD scenarios are commonly organized into feature files, which are plain text files containing a collection of related scenarios for a specific feature. Feature files are written in a human-readable language (such as Gherkin syntax) that allows easy collaboration and understanding.
Example Gherkin Syntax (Cucumber):
Feature: User Authentication
Scenario: User provides valid credentials
Given the user is on the login page
When the user enters valid username and password
Then they should be logged in successfully
Scenario: User provides invalid credentials
Given the user is on the login page
When the user enters invalid username and password
Then they should see an error message
In this example, scenarios describe the expected behavior of a user authentication feature using Gherkin syntax.
Behavior-Driven Development aims to bridge the communication gap between technical and non-technical stakeholders, ensuring that everyone involved in the project has a shared understanding of the software's behavior and requirements.
What Is The Difference Between TDD vs BDD?
Aspect | Test-Driven Development (TDD) | Behavior-Driven Development (BDD) |
Focus and perspective | Implementation of code functionality through a test-first approach | Collaboration, shared understanding, and validation of system behavior from a user's perspective |
Terminology and readability | Test cases written with programming-centric terminology | Scenarios written in a natural language format, easily understood by both technical and non-technical team members |
Collaboration and communication | Collaboration between developers and testers | Collaboration among developers, testers, and business stakeholders to define and validate system behavior |
Level of abstraction | Focuses on low-level unit tests that verify the behavior of individual code units | Focuses on higher-level tests that simulate user interactions or end-to-end scenarios |
Test organization | Tests organized based on code structure and hierarchical or modular approach | Scenarios organized around desired behavior, typically grouped by specific features or functionalities |
Source: https://katalon.com/resources-center/blog/tdd-vs-bdd
What Is The Difference Between TDD vs BDD?
Aspect | Test-Driven Development (TDD) | Behavior-Driven Development (BDD) |
Purpose | Ensures code correctness through automated tests | Promotes shared understanding, communication, and validation of system behavior |
Development workflow | Tests are written before implementing corresponding code | Scenarios are defined collaboratively before implementing the code. Can implement TDD within BDD |
Test scope | Narrow scope, typically focusing on individual code units | Broad scope, covering multiple units of code working together |
Test case style | Technical and implementation-centric | User-focused and behavior-centric |
Test granularity | Fine-grained, testing individual code units in isolation | Coarser-grained, testing system behavior as a whole |
Iterative refinement and feedback | Iteratively refines code through test failures and subsequent code modifications | Iteratively refines scenarios and behavior through collaboration and feedback |
Access Assignment on GitHub Classroom
Open Assignment on BLEND. Accept assignment at Githu classroom URL
or, Go to https://classroom.github.com/a/U-BzgV67
Install GitHub classroom extension
gh extension install github/gh-classroom
Enter your team number as in the Trello name. First person to login for each team will create the team
Team71 Team72 Team73
Team74 Team75
List your classrooms:
gh classroom list
List the assignments for a specific classroom:
gh classroom assignments
View information for a specific assignment:
gh classroom assignment
Catch2 Overview
Catch2 is a modern C++ testing framework widely used in the C++ community for unit testing. It is designed to be easy to use, expressive, and efficient. Catch2 follows a BDD (Behavior-Driven Development) style, making it readable and natural for specifying and writing tests. Catch2 is It's popular for projects of various sizes and complexities.
Here are some key features and aspects of Catch2:
Key Features:
Originally Catch2 was implemented to be extremely lightweight and simple to use
Version 2: Was a self contained header file that used a single #include “catch.hpp”
Over time this approach slowed the compile phase and the header file begin enormous
Version 3: Implemented a streamlined header file #include “catch_amalgamated.hpp” and required a compile component either catch_amalgamated.cpp or a library of module with specific functions.
Note: Our Lab uses Version 3 approach: catch_amalgamated.
Applying Catch2
In this example, the TEST_CASE macro defines a test case, and the REQUIRE and REQUIRE_FALSE macros are used for assertions. The SECTION macro allows you to further structure your test case.
#define CATCH_CONFIG_MAIN // Provides main() function
#include "catch.hpp"
// Test Fixture (Setup)
TEST_CASE("Example Test Case") {
MyClass obj; // Instantiate class under test
// Act (Execution)
int result = obj.addFunction(2, 3);
// Assert (Verification)
REQUIRE(result == 5);
REQUIRE(result == 4+1);
}
Getting Started with Catch2:
==================================================
All tests passed (2 assertions in 1 test case)
Applying Catch2
In this example, the TEST_CASE macro defines a test case, and the REQUIRE and REQUIRE_FALSE macros are used for assertions. The SECTION macro allows you to further structure your test case.
#define CATCH_CONFIG_MAIN // Provides main() function
#include "catch.hpp"
// Test Fixture (Setup)
TEST_CASE("Example Test Case") {
MyClass obj; // Instantiate class under test
// Act (Execution)
int result = obj.addFunction(2, 3);
// Assert (Verification)
REQUIRE(result == 5);
REQUIRE(result == 1);
}
Getting Started with Catch2:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.out is a Catch2 v3.4.0 host application.
Run with -? for options
-------------------------------------------------
Addition
-------------------------------------------------
test_main.cpp:20
.................................................
test_main.cpp:22: FAILED:
REQUIRE( add(-1, 1) == 5 )
with expansion:
0 == 5
=============================================
All tests passed (2 assertions in 1 test cases)
==============================================
test cases: 1 | 1 failed
assertions: 2 | 1 passed | 1 failed
Building code for test
Building code for test.
For version 2 simply
#include “catch.hpp”
Then
#define CATCH_CONFIG_MAIN
To tell catch to provide a main for testing
For version 3 simply
#include “catch.hpp”
Catch will provide its own main,
unless you tell it not to with:
#define CATCH_AMALGAMATED_CUSTOM_MAIN
#include <iostream>
#include "catch_amalgamated.hpp"
#include "../src/fac.h"
using namespace std;
#ifdef CATCH_AMALGAMATED_CUSTOM_MAIN
int main( int argc, char* argv[] ) {
// global setup...
int result = Catch::Session().run( argc, argv );
// global clean-up…
cout << "Hello Catch2 Build with custom main()\n";
return result;
}
#else //Not CATCH_AMALGAMATED_CUSTOM_MAIN
TEST_CASE("Quick Catch2 Factorial test", "[Factorial]")
{
cout << "Hello Catch2 Build with Catch2 main()\n";
cout << "Running tests on Factorial" << endl;
REQUIRE(Factorial(1) == 1);
REQUIRE(Factorial(2) == 2);
REQUIRE(Factorial(3) == 6);
REQUIRE(Factorial(4) == 24);
REQUIRE(Factorial(5) == 1);
}
#endif //ifndef CATCH_AMALGAMATED_CUSTOM_MAIN
Mocks and Stubs - Writing Tests First
Example Stub
// factorial.cpp
// Stub implementation of Factorial
int FactorialStub(int n) {
// For testing purposes, always return n * 2
return n * 2;
}
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
#include "factorial.hpp"
TEST_CASE("Factorial Tests with Stub", "[Factorial]")
{
SECTION("Factorial of 1 and 2 using Stub") {
// Use the stub implementation within this section
int result = FactorialStub(1);
REQUIRE(result == 2);
result = FactorialStub(2);
REQUIRE(result == 4);
}
SECTION("Factorial of 3 and 4 using Stub") {
// Use the stub implementation within this section
int result = FactorialStub(3);
REQUIRE(result == 6);
result = FactorialStub(4);
REQUIRE(result == 8);
}
SECTION("Factorial of 5 using Stub") {
// Use the stub implementation within this section
int result = FactorialStub(5);
REQUIRE(result == 10);
}
}
// factorial.hpp
#pragma once
int Factorial(int n);
// factorial.cpp
#include "factorial.hpp"
int Factorial(int n) {
return n <= 1 ? 1 : n * Factorial(n - 1);
}
Initially
Later
TEST_CASE and TEST_CASE_METHOD
In Catch2, both TEST_CASE and TEST_CASE_METHOD are used to define test cases,, but they are associated with different styles of testing: free functions and member functions, respectively.
TEST_CASE:
The TEST_CASE macro is used to define a test case with a free function. A free function is a standalone function that is not a member of a class. Here's an example:
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
// Free function test case
TEST_CASE("Free Function Test Case") {
// Test code goes here
REQUIRE(1 + 1 == 2);
}
TEST_CASE_METHOD:
The TEST_CASE_METHOD macro is used to define a test case with a member function. A member function is a function that is a part of a class. This is useful when you want to share setup and teardown logic across multiple test cases in the same class. Here's an example:
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
class MyTestClass {
public:
// Member function test case
void testMethod() {
// Test code goes here
REQUIRE(1 + 1 == 2);
}
};
TEST_CASE_METHOD(MyTestClass, "Member Function Test Case") {
testMethod(); // Calling the member function
}
Advanced Features: Test Cases and Sections
In Catch2, test cases and sections provide a structured way to organize and group your tests. They allow you to group related tests and share common setup and teardown logic. Here's an overview of Catch2 test cases and sections:
Test Cases:
A test case in Catch2 is a container for a set of related tests. It is defined using the TEST_CASE macro. Each test case can include one or more individual test scenarios (test cases) that are written within the body of the test case. Test cases help organize and structure your test suite.
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
// Free function test case
TEST_CASE("Free Function Test Case") {
// Test code goes here
REQUIRE(1 + 1 == 2);
}
Syntax:
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
TEST_CASE("Name of the Test Case") {
// Test scenarios go here
// ...
}
Advanced Features: Test Cases and Sections
Sections:
Sections in Catch2 allow you to further subdivide a test case into smaller, focused units. Each section can have its own setup, execution, and assertions. If a section fails, the remaining sections within the same test case are still executed. Sections are created using the SECTION macro.
Syntax:
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
TEST_CASE("Name of the Test Case") {
SECTION("Name of the Section") {
// Test scenarios for this section
// ...
}
SECTION("Another Section") {
// Test scenarios for another section
// ...
}
}
Advanced Features: Test Cases and Sections
In this example, the Math Operations test case contains two sections, each testing a different math operation. Sections are used to further organize and structure the tests within a test case.
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
TEST_CASE("Math Operations") {
SECTION("Addition") {
REQUIRE(1 + 1 == 2);
}
SECTION("Subtraction") {
REQUIRE(4 - 2 == 2);
}
}
In this example, the String Operations test case contains two sections, each testing a different aspect of string operations. Using sections helps maintain a clear and organized structure within the test case.
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
TEST_CASE("String Operations") {
SECTION("Concatenation") {
std::string str1 = "Hello";
std::string str2 = "World";
REQUIRE(str1 + str2 == "HelloWorld");
}
SECTION("Substring") {
std::string fullString = "Catch2 Sections";
REQUIRE(fullString.substr(0, 6) == "Catch2");
}
}
Test cases and sections provide a way to structure your tests logically, making it easier to manage and understand the behavior being tested. They are particularly useful when you have multiple scenarios or variations to test within the same context.
Grouping Tests and Structuring Test Cases
Grouping Tests and Structuring Test Cases:
In summary, test cases and sections provide a flexible and expressive way to structure your tests in Catch2, making it easier to manage and understand the behavior being tested. They contribute to a clean and organized test suite, which is crucial for maintaining and evolving your codebase.
Basic Assertions
REQUIRE and CHECK:
REQUIRE(expression);
CHECK(expression);
REQUIRE_FALSE and CHECK_FALSE:
REQUIRE_FALSE(expression);
CHECK_FALSE(expression);
REQUIRE_THROWS and CHECK_THROWS:
REQUIRE_THROWS(expression);
CHECK_THROWS(expression);
REQUIRE_NOTHROW and CHECK_NOTHROW:
REQUIRE_NOTHROW(expression);
CHECK_NOTHROW(expression);
REQUIRE_THAT and CHECK_THAT:
REQUIRE_THAT(variable, matcher);
CHECK_THAT(variable, matcher);
REQUIRE_THAT and CHECK_THAT with Lambda Expressions:
REQUIRE_THAT(variable, Catch::Matchers::Predicate([](auto v){ return v > 0; }, "positive value"));
CHECK_THAT(variable,
Catch::Matchers::All([](auto v){ return v > 0; }));
In Catch2, assertions are statements that express the expected behavior of your code and are used to validate conditions during testing.
Here are some common basic assertions:
Example of Basic Assertions
#define CATCH_CONFIG_MAIN
#include "catch.hpp"
TEST_CASE("Basic Assertions") {
int value = 42;
std::vector<int> numbers = {1, 2, 3, 4, 5};
SECTION("Equality") {
REQUIRE(value == 42);
CHECK(value != 100);
}
SECTION("Approximate Matchers") {
REQUIRE(Approx(3.14) == 3.14159);
CHECK_FALSE(Approx(1.0) == 1.01);
}
SECTION("Container Matchers") {
REQUIRE_THAT(numbers, VectorContains(2, 4));
CHECK_THAT(numbers, !Contains(10));
}
. . .
. . .
SECTION("String Matchers") {
std::string text = "Catch2 is awesome!";
REQUIRE_THAT(text, StartsWith("Catch2"));
CHECK_THAT(text, EndsWith("awesome!"));
}
SECTION("Custom Assertions") {
REQUIRE(value > 0);
CHECK_THAT(numbers,
Catch::Matchers::All([](int n){
return n > 0; })
);
}
}
Custom Assertions:
Custom Assertion Macro:
#define REQUIRE_EVEN(value) REQUIRE(value % 2 == 0)
#define CHECK_ODD(value) CHECK(value % 2 != 0)
TEST_CASE("Custom Assertions") {
int evenNumber = 10;
int oddNumber = 7;
SECTION("Even and Odd Numbers") {
REQUIRE_EVEN(evenNumber);
CHECK_ODD(oddNumber);
}
}
Custom Assertion Function:
template <typename T>
void require_positive(const T& value) {
REQUIRE(value > 0);
}
template <typename T>
void check_negative(const T& value) {
CHECK(value < 0);
}
TEST_CASE("Custom Assertion Functions") {
int positiveValue = 42;
int negativeValue = -7;
SECTION("Positive and Negative Numbers") {
require_positive(positiveValue);
check_negative(negativeValue);
}
}
In addition to the built-in assertions, Catch2 allows you to create custom assertions using macros or functions. Custom assertions can be useful for encapsulating complex or repetitive validation logic.
Custom assertions can improve:
Use custom assertions when you find yourself repeating the same validation patterns across multiple test cases.
Best Practices
1. Isolate Tests:
2. Mocking and Stubbing:
Unit testing is a crucial aspect of software development, and adhering to best practices ensures that your tests are effective, maintainable, and provide accurate feedback about the health of your codebase. When it comes to isolation and independence in unit testing, here are some best practices:
Best Practices - con’t
3. Avoid Global State:
4. Use Dependency Injection:
5. Avoid Hard-Coding Values:
Best Practices - con’t
6. Keep Tests Fast and Deterministic:
7. Separate Test and Production Code:
8. Avoid Testing Private Methods:
9. Use Descriptive Test Names:
Best Practices - con’t
10. Regularly Refactor Tests:
By following these best practices, you ensure that your unit tests are isolated, independent, and provide reliable feedback about the correctness of your code. This, in turn, contributes to a more robust and maintainable codebase.
END