Home Explore Blog CI



nixpkgs

12th chunk of `doc/languages-frameworks/python.section.md`
985be5e26ae71db5ebe2fbadc6456f7c7e3577762d6c4c900000000100000fae
1. We begin with importing the Nix Packages collections. `import <nixpkgs>`
   imports the `<nixpkgs>` function, `{}` calls it and the `with` statement
   brings all attributes of `nixpkgs` in the local scope. These attributes form
   the main package set.
2. Then we create a Python 3.12 environment with the [`withPackages`](#python.withpackages-function) function, as before.
3. The [`withPackages`](#python.withpackages-function) function expects us to provide a function as an argument
   that takes the set of all Python packages and returns a list of packages to
   include in the environment. Here, we select the packages `numpy` and `toolz`
   from the package set.

To combine this with `mkShell` you can:

```nix
with import <nixpkgs> { };
let
  pythonEnv = python312.withPackages (ps: [
    ps.numpy
    ps.toolz
  ]);
in
mkShell {
  packages = [
    pythonEnv

    black
    mypy

    libffi
    openssl
  ];
}
```

This will create a unified environment that has not just our Python interpreter
and its Python dependencies, but also tools like `black` or `mypy` and libraries
like `libffi` the `openssl` in scope. This is generic and can span any number of
tools or languages across the Nixpkgs ecosystem.

##### Installing environments globally on the system {#installing-environments-globally-on-the-system}

Up to now, we've been creating environments scoped to an ad-hoc shell session,
or a single script, or a single project. This is generally advisable, as it
avoids pollution across contexts.

However, sometimes we know we will often want a Python with some basic packages,
and want this available without having to enter into a shell or build context.
This can be useful to have things like vim/emacs editors and plugins or shell
tools "just work" without having to set them up, or when running other software
that expects packages to be installed globally.

To create your own custom environment, create a file in `~/.config/nixpkgs/overlays/`
that looks like this:

```nix
# ~/.config/nixpkgs/overlays/myEnv.nix
self: super: {
  myEnv = super.buildEnv {
    name = "myEnv";
    paths = [
      # A Python 3 interpreter with some packages
      (self.python3.withPackages (
        ps: with ps; [
          pyflakes
          pytest
          black
        ]
      ))

      # Some other packages we'd like as part of this env
      self.mypy
      self.black
      self.ripgrep
      self.tmux
    ];
  };
}
```

You can then build and install this to your profile with:

```sh
nix-env -iA myEnv
```

One limitation of this is that you can only have 1 Python env installed
globally, since they conflict on the `python` to load out of your `PATH`.

If you get a conflict or prefer to keep the setup clean, you can have `nix-env`
atomically *uninstall* all other imperatively installed packages and replace
your profile with just `myEnv` by using the `--replace` flag.

##### Environment defined in `/etc/nixos/configuration.nix` {#environment-defined-in-etcnixosconfiguration.nix}

For the sake of completeness, here's how to install the environment system-wide
on NixOS.

```nix
{
  # ...

  environment.systemPackages = with pkgs; [
    (python310.withPackages (
      ps: with ps; [
        numpy
        toolz
      ]
    ))
  ];
}
```

### Developing with Python {#developing-with-python}

Above, we were mostly just focused on use cases and what to do to get started
creating working Python environments in nix.

Now that you know the basics to be up and running, it is time to take a step
back and take a deeper look at how Python packages are packaged on Nix.

#### Python library packages in Nixpkgs {#python-library-packages-in-nixpkgs}

With Nix all packages are built by functions. The main function in Nix for
building Python libraries is [`buildPythonPackage`](#buildpythonpackage-function). Let's see how we can build the
`toolz` package.

```nix
{
  lib,
  buildPythonPackage,
  fetchPypi,
  setuptools,
}:

buildPythonPackage rec {
  pname = "toolz";
  version = "0.10.0";

Title: Combining Environments with mkShell, Installing Globally, and Python Package Development in Nixpkgs
Summary
This section demonstrates combining Python environments with other tools and libraries using `mkShell`. It also explains how to install custom environments globally on the system by creating overlays in `~/.config/nixpkgs/overlays/`. The guide then touches on installing environments system-wide on NixOS by modifying `/etc/nixos/configuration.nix`. Finally, it introduces `buildPythonPackage`, the primary function in Nixpkgs for building Python library packages, using `toolz` as an example.