Home Explore Blog CI



nixpkgs

16th chunk of `doc/languages-frameworks/python.section.md`
71051ab6d8982b5d1c35c22f0e0146919e3d73dd95e8d5460000000100000fb2
    changelog = "https://github.com/pyFFTW/pyFFTW/releases/tag/v${version}";
    description = "Pythonic wrapper around FFTW, the FFT library, presenting a unified interface for all the supported transforms";
    homepage = "http://hgomersall.github.com/pyFFTW";
    license = with lib.licenses; [
      bsd2
      bsd3
    ];
  };
}
```

Note also the line [`doCheck = false;`](#var-stdenv-doCheck), we explicitly disabled running the test-suite.

#### Testing Python Packages {#testing-python-packages}

It is highly encouraged to have testing as part of the package build. This
helps to avoid situations where the package was able to build and install,
but is not usable at runtime.
Your package should provide its own [`checkPhase`](#ssec-check-phase).

::: {.note}
The [`checkPhase`](#ssec-check-phase) for python maps to the `installCheckPhase` on a
normal derivation. This is due to many python packages not behaving well
to the pre-installed version of the package. Version info, and natively
compiled extensions generally only exist in the install directory, and
thus can cause issues when a test suite asserts on that behavior.
:::

::: {.note}
Tests should only be disabled if they don't agree with nix
(e.g. external dependencies, network access, flakey tests), however,
as many tests should be enabled as possible. Failing tests can still be
a good indication that the package is not in a valid state.
:::

::: {.note}
We only want to test the functionality of a package. In particular, we are not
interested in coverage, formatting, and type checking. If pytest fails with
`unrecognized arguments: --cov`, add `pytest-cov-stub` to `nativeCheckInputs`
rather than `pytest-cov`.
:::

#### Using pytest {#using-pytest}

Pytest is the most common test runner for python repositories. A trivial
test run would be:

```nix
{
  nativeCheckInputs = [ pytest ];
  checkPhase = ''
    runHook preCheck

    pytest

    runHook postCheck
  '';
}
```

However, many repositories' test suites do not translate well to nix's build
sandbox, and will generally need many tests to be disabled.

This is achievable by
- Including paths or test items (`path/to/file.py::MyClass` or `path/to/file.py::MyClass::test_method`) with positional arguments.
- Excluding paths with `--ignore` or globbed paths with `--ignore-glob`.
- Excluding test items using the `--deselect` flag.
- Including or excluding classes or test methods by their name using the `-k` flag.
- Including or excluding test by their marks using the `-m` flag.

We highly recommend `pytestCheckHook` for an easier and more structural setup.

#### Using pytestCheckHook {#using-pytestcheckhook}

`pytestCheckHook` is a convenient hook which will set up (or configure)
a [`checkPhase`](#ssec-check-phase) to run `pytest`. This is also beneficial
when a package may need many items disabled to run the test suite.
Most packages use `pytest` or `unittest`, which is compatible with `pytest`,
so you will most likely use `pytestCheckHook`.

To use `pytestCheckHook`, add it to `nativeCheckInputs`.
Adding `pytest` is not required, since it is included with `pytestCheckHook`.

```nix
{
  nativeCheckInputs = [
    pytestCheckHook
  ];
}
```

`pytestCheckHook` recognizes the following attributes:

`enabledTestPaths` and `disabledTestPaths`

:   To specify path globs (files or directories) or test items.

`enabledTests` and `disabledTests`

:   To specify keywords for class names or test method names.

`enabledTestMarks` and `disabledTestMarks`

:   To specify test marks.

`pytestFlags`

:   To append additional command-line arguments to `pytest`.

By default, `pytest` automatically discovers which tests to run.
If tests are explicitly enabled, only those tests will run.
A test, that is both enabled and disabled, will not run.

The following example demonstrates usage of various `pytestCheckHook` attributes:

```nix
{
  nativeCheckInputs = [
    pytestCheckHook
  ];

  # Allow running the following test paths and test objects.
  enabledTestPaths = [

Title: Testing Python Packages with pytest and pytestCheckHook
Summary
This section emphasizes the importance of testing Python packages in Nix and discourages disabling tests unless absolutely necessary (e.g., external dependencies, flaky tests). It describes how to use pytest as a test runner and introduces `pytestCheckHook` as a convenient hook to manage pytest configurations and disable specific tests. It also warns against focusing on coverage, formatting, and type checking. Several attributes of `pytestCheckHook` such as `enabledTestPaths`, `disabledTestPaths`, `enabledTests`, `disabledTests`, `enabledTestMarks`, `disabledTestMarks` and `pytestFlags` are introduced.