Home Explore Blog Models CI



nixpkgs

13th chunk of `doc/languages-frameworks/python.section.md`
e7fededb3708ac5593d492c7eeb22279b56d6691eea919ae0000000100000fa0
creating working Python environments in nix.

Now that you know the basics to be up and running, it is time to take a step
back and take a deeper look at how Python packages are packaged on Nix.

#### Python library packages in Nixpkgs {#python-library-packages-in-nixpkgs}

With Nix all packages are built by functions. The main function in Nix for
building Python libraries is [`buildPythonPackage`](#buildpythonpackage-function). Let's see how we can build the
`toolz` package.

```nix
{
  lib,
  buildPythonPackage,
  fetchPypi,
  setuptools,
}:

buildPythonPackage rec {
  pname = "toolz";
  version = "0.10.0";
  pyproject = true;

  src = fetchPypi {
    inherit pname version;
    hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";
  };

  build-system = [ setuptools ];

  # has no tests
  doCheck = false;

  pythonImportsCheck = [
    "toolz.itertoolz"
    "toolz.functoolz"
    "toolz.dicttoolz"
  ];

  meta = {
    changelog = "https://github.com/pytoolz/toolz/releases/tag/${version}";
    homepage = "https://github.com/pytoolz/toolz";
    description = "List processing tools and functional utilities";
    license = lib.licenses.bsd3;
  };
}
```

What happens here? The function [`buildPythonPackage`](#buildpythonpackage-function) is called and as argument
it accepts a set. In this case the set is a recursive set, `rec`. One of the
arguments is the name of the package, which consists of a basename (generally
following the name on PyPI) and a version. Another argument, `src` specifies the
source, which in this case is fetched from PyPI using the helper function
`fetchPypi`. The argument `doCheck` is used to set whether tests should be run
when building the package. Since there are no tests, we rely on [`pythonImportsCheck`](#using-pythonimportscheck)
to test whether the package can be imported. Furthermore, we specify some meta
information. The output of the function is a derivation.

An expression for `toolz` can be found in the Nixpkgs repository. As explained
in the introduction of this Python section, a derivation of `toolz` is available
for each interpreter version, e.g. `python313.pkgs.toolz` refers to the `toolz`
derivation corresponding to the CPython 3.13 interpreter.

The above example works when you're directly working on
`pkgs/top-level/python-packages.nix` in the Nixpkgs repository. Often though,
you will want to test a Nix expression outside of the Nixpkgs tree.

The following expression creates a derivation for the `toolz` package,
and adds it along with a `numpy` package to a Python environment.

```nix
with import <nixpkgs> { };

(
  let
    my_toolz = python313.pkgs.buildPythonPackage rec {
      pname = "toolz";
      version = "0.10.0";
      pyproject = true;

      src = fetchPypi {
        inherit pname version;
        hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";
      };

      build-system = [ python313.pkgs.setuptools ];

      # has no tests
      doCheck = false;

      meta = {
        homepage = "https://github.com/pytoolz/toolz/";
        description = "List processing tools and functional utilities";
        # [...]
      };
    };

  in
  python313.withPackages (
    ps: with ps; [
      numpy
      my_toolz
    ]
  )
).env
```

Executing `nix-shell` will result in an environment in which you can use
Python 3.13 and the `toolz` package. As you can see we had to explicitly mention
for which Python version we want to build a package.

So, what did we do here? Well, we took the Nix expression that we used earlier
to build a Python environment, and said that we wanted to include our own
version of `toolz`, named `my_toolz`. To introduce our own package in the scope
of [`withPackages`](#python.withpackages-function) we used a `let` expression. You can see that we used
`ps.numpy` to select numpy from the nixpkgs package set (`ps`). We did not take
`toolz` from the Nixpkgs package set this time, but instead took our own version
that we introduced with the `let` expression.


Title: Packaging Python Libraries with buildPythonPackage and Custom Derivations
Summary
This chunk elaborates on packaging Python libraries in Nix, focusing on the `buildPythonPackage` function. It provides a detailed example using the `toolz` package, explaining key attributes such as `pname`, `version`, `src` (fetched via `fetchPypi`), `build-system`, test mechanisms (`doCheck`, `pythonImportsCheck`), and `meta` information. The text clarifies that this function produces a derivation, and pre-existing derivations are available in Nixpkgs for various Python interpreter versions (e.g., `python313.pkgs.toolz`). Furthermore, it demonstrates how to define a custom package derivation outside the main Nixpkgs tree using a `let` expression and integrate it into a specific Python environment created with `python313.withPackages`, highlighting the necessity of specifying the target Python version when building custom packages.