Home Explore Blog CI



nixpkgs

4th chunk of `doc/build-helpers/testers.chapter.md`
50092ad48ce29bfba300023998087c92b0fe683e1ef995bc0000000100000faa
This example will run the command `hello --version`, and then check that the version of the `hello` package is in the output of the command.

```nix
{
  passthru.tests.version = testers.testVersion { package = hello; };
}
```

:::

:::{.example #ex-testversion-different-commandversion}

# Check the program version using a specified command and expected version string

This example will run the command `leetcode -V`, and then check that `leetcode 0.4.2` is in the output of the command as a whole word (separated by whitespaces).
This means that an output like "leetcode 0.4.21" would fail the tests, and an output like "You're running leetcode 0.4.2" would pass the tests.

A common usage of the `version` attribute is to specify `version = "v${version}"`.

```nix
{
  version = "0.4.2";

  passthru.tests.version = testers.testVersion {
    package = leetcode-cli;
    command = "leetcode -V";
    version = "leetcode ${version}";
  };
}
```

:::

## `testBuildFailure` {#tester-testBuildFailure}

Make sure that a build does not succeed. This is useful for testing testers.

This returns a derivation with an override on the builder, with the following effects:

 - Fail the build when the original builder succeeds
 - Move `$out` to `$out/result`, if it exists (assuming `out` is the default output)
 - Save the build log to `$out/testBuildFailure.log` (same)

While `testBuildFailure` is designed to keep changes to the original builder's environment to a minimum, some small changes are inevitable:

 - The file `$TMPDIR/testBuildFailure.log` is present. It should not be deleted.
 - `stdout` and `stderr` are a pipe instead of a tty. This could be improved.
 - One or two extra processes are present in the sandbox during the original builder's execution.
 - The derivation and output hashes are different, but not unusual.
 - The derivation includes a dependency on `buildPackages.bash` and `expect-failure.sh`, which is built to include a transitive dependency on `buildPackages.coreutils` and possibly more.
   These are not added to `PATH` or any other environment variable, so they should be hard to observe.

:::{.example #ex-testBuildFailure-showingenvironmentchanges}

# Check that a build fails, and verify the changes made during build

```nix
runCommand "example"
  {
    failed = testers.testBuildFailure (
      runCommand "fail" { } ''
        echo ok-ish >$out
        echo failing though
        exit 3
      ''
    );
  }
  ''
    grep -F 'ok-ish' $failed/result
    grep -F 'failing though' $failed/testBuildFailure.log
    [[ 3 = $(cat $failed/testBuildFailure.exit) ]]
    touch $out
  ''
```

:::

## `testBuildFailure'` {#tester-testBuildFailurePrime}

This tester wraps the functionality provided by [`testers.testBuildFailure`](#tester-testBuildFailure) to make writing checks easier by simplifying checking the exit code of the builder and asserting the existence of entries in the builder's log.
Additionally, users may specify a script containing additional checks, accessing the result of applying `testers.testBuildFailure` through the variable `failed`.

NOTE: This tester will produce an empty output and exit with success if none of the checks fail; there is no need to `touch "$out"` in `script`.

:::{.example #ex-testBuildFailurePrime-doc-example}

# Check that a build fails, and verify the changes made during build

Re-using the example from [`testers.testBuildFailure`](#ex-testBuildFailure-showingenvironmentchanges), we can see how common checks are made easier and remove the need for `runCommand`:

```nix
testers.testBuildFailure' {
  drv = runCommand "doc-example" { } ''
    echo ok-ish >"$out"
    echo failing though
    exit 3
  '';
  expectedBuilderExitCode = 3;
  expectedBuilderLogEntries = [ "failing though" ];
  script = ''
    grep --silent -F 'ok-ish' "$failed/result"
  '';
}
```

:::

### Inputs {#tester-testBuildFailurePrime-inputs}

`drv` (derivation)

: The failing derivation to wrap with `testBuildFailure`.

`name` (string, optional)

Title: Detailed Explanation of `testBuildFailure` and `testBuildFailure'` Testers in Nixpkgs
Summary
This section describes the `testBuildFailure` tester, which ensures that a build fails, useful for testing other testers. It details how it works, including creating a derivation that overrides the builder to fail when the original succeeds, saving the log, and moving the output. It also lists unavoidable environment changes. Then, it describes `testBuildFailure'`, which simplifies checking the exit code and log entries of a failed build, allowing users to specify a script with additional checks. An example demonstrates how `testBuildFailure'` streamlines the process of verifying build failures.