# Running tests

## Coverage analysis

A [coverage analysis report] is available online
You can build it yourself:


```
# nix build .#hydraJobs.coverage
# xdg-open ./result/coverage/index.html
```

[Extensive records of build metrics](https://hydra.nixos.org/job/nix/master/coverage#tabs-charts), such as test coverage over time, are also available online.

## Unit-tests

The unit tests are defined using the [googletest] and [rapidcheck] frameworks.


### Source and header layout

> An example of some files, demonstrating much of what is described below
>
> ```
> src
> ├── libexpr
> │   ├── meson.build
> │   ├── include/nix/expr/value/context.hh
> │   ├── value/context.cc
> │   …
> │
> ├── tests
> │   │
> │   …
> │   ├── libutil-tests
> │   │   ├── meson.build
> │   │   …
> │   │   └── data
> │   │       ├── git/tree.txt
> │   │       …
> │   │
> │   ├── libexpr-test-support
> │   │   ├── meson.build
> │   │   ├── include/nix/expr
> │   │   │   ├── meson.build
> │   │   │   └── tests
> │   │   │       ├── value/context.hh
> │   │   │       …
> │   │   └── tests
> │   │       ├── value/context.cc
> │   │       …
> │   │
> │   ├── libexpr-tests
> │   …   ├── meson.build
> │       ├── value/context.cc
> │       …
> …
> ```

The tests for each Nix library (`libnixexpr`, `libnixstore`, etc..) live inside a directory `src/${library_name_without-nix}-test`.
Given an interface (header) and implementation pair in the original library, say, `src/libexpr/include/nix/expr/value/context.hh` and `src/libexpr/value/context.cc`, we write tests for it in `src/libexpr-tests/value/context.cc`, and (possibly) declare/define additional interfaces for testing purposes in `src/libexpr-test-support/include/nix/expr/tests/value/context.hh` and `src/libexpr-test-support/tests/value/context.cc`.

Data for unit tests is stored in a `data` subdir of the directory for each unit test executable.
For example, `libnixstore` code is in `src/libstore`, and its test data is in `src/libstore-tests/data`.
The path to the `src/${library_name_without-nix}-test/data` directory is passed to the unit test executable with the environment variable `_NIX_TEST_UNIT_DATA`.
Note that each executable only gets the data for its tests.

The unit test libraries are in `src/${library_name_without-nix}-test-support`.
All headers are in a `tests` subdirectory so they are included with `#include "nix/tests/"`.

The use of all these separate directories for the unit tests might seem inconvenient, as for example the tests are not "right next to" the part of the code they are testing.
But organizing the tests this way has one big benefit:
there is no risk of any build-system wildcards for the library accidentally picking up test code that should not built and installed as part of the library.

### Running tests

You can run the whole testsuite with `meson test` from the Meson build directory, or the tests for a specific component with `meson test nix-store-tests`.
A environment variables that Google Test accepts are also worth knowing:

1. [`GTEST_FILTER`](https://google.github.io/googletest/advanced.html#running-a-subset-of-the-tests)

   This is used for finer-grained filtering of which tests to run.


2. [`GTEST_BRIEF`](https://google.github.io/googletest/advanced.html#suppressing-test-passes)

   This is used to avoid logging passing tests.

3. [`GTEST_BREAK_ON_FAILURE`](https://google.github.io/googletest/advanced.html#turning-assertion-failures-into-break-points)

   This is used to create a debugger breakpoint when an assertion failure occurs.

Putting the first two together, one might run

```bash
GTEST_BRIEF=1 GTEST_FILTER='ErrorTraceTest.*' meson test nix-expr-tests -v
```

for short but comprensive output.

### Debugging tests

For debugging, it is useful to combine the third option above with Meson's [`--gdb`](https://mesonbuild.com/Unit-tests.html#other-test-options) flag:

```bash
GTEST_BRIEF=1 GTEST_FILTER='Group.my-failing-test' meson test nix-expr-tests --gdb
```

This will: