Test Runner 🔗

The test runner or runner is the core component responsible for discovering and executing tests. It runs test cases, reports results — including pass or fail status — and provides detailed information about any failures or errors.

In Audition, the test runner is a dedicated executable program you create specifically for testing your application. An executable becomes a test runner by linking against the Audition library. Your executable need not implement a main function. This is because Audition implements its own main function which enables it to respond to program arguments, discover tests, and run them without requiring manual setup on your part.

You can provide various command-line arguments to the test runner. For instance, you can specify an argument to control which test reporter is used or which test cases are executed. The most common arguments are documented in the running tests section. You can view all available arguments by executing the test runner with the -h or --help flag.

Suites and Test Cases 🔗

A test case or test is a set of conditions and expected outcomes used to assess whether a specific piece of code — usually a function — behaves correctly. Test cases are essential for validating software functionality and ensuring it meets specified requirements.

A test suite or suite is a collection of test cases that are grouped together for execution. Suites organize tests and allow for the association of fixtures as we'll discuss shortly.

In Audition, a test suite and test case are declared together using the TEST directive. In the following code snippet a new test case is defined with the name myFirstTest belonging to the test suite mySuite. Note that the test suite mySuite is implicitly created the first time a test is defined that references it.

TEST(mySuite, myFirstTest) {
    // ...
}

If another test case is defined belonging to the same suite, as shown in the snippet below, then it is implicitly appended to the existing suite. A test suite can have any number of test cases associated with it, but each test case must have a unique name within that suite.

TEST(mySuite, mySecondTest) {
    // ...
}

Audition automatically discovers test cases for you and will execute them in the order they are declared in your C source file. Test suites are run in alphabetical order. Together, these rules ensure a predictable test execution order.

Audition will always execute a test suite in its entirety before executing tests belonging to another test suite. This is true regardless of whether the test suite has tests declared across multiple source files.

By default, Audition executes all tests and suites, but you can filter which tests and suites are executed by passing the appropriate program arguments to the test runner.

Fixtures 🔗

Fixtures prepare the test environment. They ensure that tests run with the expected initial state, remain independent of each other, and that test resources are released afterward. Fixtures are crucial for maintaining the reliability and predictability of test outcomes.

Fixtures involve two main phases: setup and teardown.

• The setup phase prepares the necessary context (such as initializing data structures or creating test data) before a test runs.
• The teardown phase cleans up any resources used during a test (like releasing memory or resetting global flags).

Audition supports three types of fixtures: test fixtures, suite fixtures, and runner fixtures. Test fixtures are run before and after each individual test case, suite fixtures are run before and after each test suite, and runner fixtures are run before and after all test suites.

Fixtures are explored further, with code examples, in the writing fixtures guide.

Assertions 🔗

Assertions are used to validate the output of a function against an expected value, ensuring that your code behaves correctly in various scenarios. For example, an assertion can check if two strings are identical or if one floating-point value is greater than another within some tolerance.

In Audition, assertions are grouped into two categories: those that can abort the test case and those that cannot.

• Assertions that can abort the test case will halt execution of the test if the assertion fails.
• Assertions that cannot abort the test case will flag the test as failed but continue executing it.

In Audition, assertions that can abort the test case begin with ASSERT, while those that cannot abort it begin with EXPECT. For example, ASSERT_EQ can abort the test, but EXPECT_EQ cannot. A complete list of available assertions is documented here.

Mocks 🔗

Mocking simulates the behavior of real functions in a controlled way. Mocks allow you to isolate the unit of code being tested, ensuring that tests focus solely on that unit’s functionality without being affected by external dependencies. This is particularly useful for isolating code from external systems, such as databases or networked services, and testing error conditions.

Audition allows for three categories of mocks:

• Stubs - Provide predefined responses to a specific function wihtout calling the original.
• Fakes - Provide a simplified implementation of a function, optionally calling the original.
• Spies - Records information about a function call and calls through to the original.

Audition provides first-party support for stubs and fakes. Spies can be implemented yourself using fake functions.