Home Explore Blog Models CI



nixpkgs

12th chunk of `doc/languages-frameworks/python.section.md`
e6499cfe47d3ae6d0edc60e1ac723957418e138055e34e800000000100000fbe
3. The [`withPackages`](#python.withpackages-function) function expects us to provide a function as an argument
   that takes the set of all Python packages and returns a list of packages to
   include in the environment. Here, we select the packages `numpy` and `toolz`
   from the package set.

To combine this with `mkShell` you can:

```nix
with import <nixpkgs> { };
let
  pythonEnv = python313.withPackages (ps: [
    ps.numpy
    ps.toolz
  ]);
in
mkShell {
  packages = [
    pythonEnv

    black
    mypy

    libffi
    openssl
  ];
}
```

This will create a unified environment that has not just our Python interpreter
and its Python dependencies, but also tools like `black` or `mypy` and libraries
like `libffi` and `openssl` in scope. This is generic and can span any number of
tools or languages across the Nixpkgs ecosystem.

##### Installing environments globally on the system {#installing-environments-globally-on-the-system}

Up to now, we've been creating environments scoped to an ad-hoc shell session,
or a single script, or a single project. This is generally advisable, as it
avoids pollution across contexts.

However, sometimes we know we will often want a Python with some basic packages,
and want this available without having to enter into a shell or build context.
This can be useful to have things like vim/emacs editors and plugins or shell
tools "just work" without having to set them up, or when running other software
that expects packages to be installed globally.

To create your own custom environment, create a file in `~/.config/nixpkgs/overlays/`
that looks like this:

```nix
# ~/.config/nixpkgs/overlays/myEnv.nix
self: super: {
  myEnv = super.buildEnv {
    name = "myEnv";
    paths = [
      # A Python 3 interpreter with some packages
      (self.python3.withPackages (
        ps: with ps; [
          pyflakes
          pytest
          black
        ]
      ))

      # Some other packages we'd like as part of this env
      self.mypy
      self.black
      self.ripgrep
      self.tmux
    ];
  };
}
```

You can then build and install this to your profile with:

```sh
nix-env -iA myEnv
```

One limitation of this is that you can only have 1 Python env installed
globally, since they conflict on the `python` to load out of your `PATH`.

If you get a conflict or prefer to keep the setup clean, you can have `nix-env`
atomically *uninstall* all other imperatively installed packages and replace
your profile with just `myEnv` by using the `--replace` flag.

##### Environment defined in `/etc/nixos/configuration.nix` {#environment-defined-in-etcnixosconfiguration.nix}

For the sake of completeness, here's how to install the environment system-wide
on NixOS.

```nix
{
  # ...

  environment.systemPackages = with pkgs; [
    (python310.withPackages (
      ps: with ps; [
        numpy
        toolz
      ]
    ))
  ];
}
```

### Developing with Python {#developing-with-python}

Above, we were mostly just focused on use cases and what to do to get started
creating working Python environments in nix.

Now that you know the basics to be up and running, it is time to take a step
back and take a deeper look at how Python packages are packaged on Nix.

#### Python library packages in Nixpkgs {#python-library-packages-in-nixpkgs}

With Nix all packages are built by functions. The main function in Nix for
building Python libraries is [`buildPythonPackage`](#buildpythonpackage-function). Let's see how we can build the
`toolz` package.

```nix
{
  lib,
  buildPythonPackage,
  fetchPypi,
  setuptools,
}:

buildPythonPackage rec {
  pname = "toolz";
  version = "0.10.0";
  pyproject = true;

  src = fetchPypi {
    inherit pname version;
    hash = "sha256-CP3V73yWSArRHBLUct4hrNMjWZlvaaUlkpm1QP66RWA=";
  };

  build-system = [ setuptools ];

  # has no tests
  doCheck = false;

  pythonImportsCheck = [
    "toolz.itertoolz"
    "toolz.functoolz"
    "toolz.dicttoolz"
  ];

  meta = {
    changelog = "https://github.com/pytoolz/toolz/releases/tag/${version}";

Title: Nixpkgs Python: Global Environments and Custom Package Building
Summary
This chunk demonstrates how to create comprehensive project development environments using `mkShell` to combine Python interpreters, packages (e.g., `numpy`, `toolz`), development tools (`black`, `mypy`), and system libraries (`libffi`, `openssl`). It then explains how to install custom Python environments globally using Nix overlays (`~/.config/nixpkgs/overlays/myEnv.nix`) and `nix-env`, noting the limitation of having only one global Python environment due to `PATH` conflicts. For NixOS users, it shows how to achieve system-wide installation via `environment.systemPackages` in `configuration.nix`. Finally, the text transitions to how Python packages are built in Nix, introducing `buildPythonPackage` as the main function and providing a detailed example of defining the `toolz` package, including `src`, `build-system`, and `pythonImportsCheck` attributes.