Home Explore Blog CI



nixpkgs

9th chunk of `doc/languages-frameworks/python.section.md`
c20fb6023c521ebd9d33b69c240b237bccea9d944b8256790000000100000fe1
In the Nixpkgs tree Python applications can be found throughout, depending on
what they do, and are called from the main package set. Python libraries,
however, are in separate sets, with one set per interpreter version.

The interpreters have several common attributes. One of these attributes is
`pkgs`, which is a package set of Python libraries for this specific
interpreter. E.g., the `toolz` package corresponding to the default interpreter
is `python3.pkgs.toolz`, and the CPython 3.12 version is `python312.pkgs.toolz`.
The main package set contains aliases to these package sets, e.g.
`pythonPackages` refers to `python.pkgs` and `python312Packages` to
`python312.pkgs`.

#### Installing Python and packages {#installing-python-and-packages}

The Nix and NixOS manuals explain how packages are generally installed. In the
case of Python and Nix, it is important to make a distinction between whether the
package is considered an application or a library.

Applications on Nix are typically installed into your user profile imperatively
using `nix-env -i`, and on NixOS declaratively by adding the package name to
`environment.systemPackages` in `/etc/nixos/configuration.nix`. Dependencies
such as libraries are automatically installed and should not be installed
explicitly.

The same goes for Python applications. Python applications can be installed in
your profile, and will be wrapped to find their exact library dependencies,
without impacting other applications or polluting your user environment.

But Python libraries you would like to use for development cannot be installed,
at least not individually, because they won't be able to find each other
resulting in import errors. Instead, it is possible to create an environment
with [`python.buildEnv`](#python.buildenv-function) or [`python.withPackages`](#python.withpackages-function) where the interpreter and other
executables are wrapped to be able to find each other and all of the modules.

In the following examples we will start by creating a simple, ad-hoc environment
with a nix-shell that has `numpy` and `toolz` in Python 3.12; then we will create
a re-usable environment in a single-file Python script; then we will create a
full Python environment for development with this same environment.

Philosophically, this should be familiar to users who are used to a `venv` style
of development: individual projects create their own Python environments without
impacting the global environment or each other.

#### Ad-hoc temporary Python environment with `nix-shell` {#ad-hoc-temporary-python-environment-with-nix-shell}

The simplest way to start playing with the way nix wraps and sets up Python
environments is with `nix-shell` at the cmdline. These environments create a
temporary shell session with a Python and a *precise* list of packages (plus
their runtime dependencies), with no other Python packages in the Python
interpreter's scope.

To create a Python 3.12 session with `numpy` and `toolz` available, run:

```sh
$ nix-shell -p 'python312.withPackages(ps: with ps; [ numpy toolz ])'
```

By default `nix-shell` will start a `bash` session with this interpreter in our
`PATH`, so if we then run:

```Python console
[nix-shell:~/src/nixpkgs]$ python3
Python 3.12.4 (main, Jun  6 2024, 18:26:44) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy; import toolz
```

Note that no other modules are in scope, even if they were imperatively
installed into our user environment as a dependency of a Python application:

```Python console
>>> import requests
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'requests'
```

We can add as many additional modules onto the `nix-shell` as we need, and we
will still get 1 wrapped Python interpreter. We can start the interpreter
directly like so:

```sh
$ nix-shell -p "python312.withPackages (ps: with ps; [ numpy toolz requests ])" --run python3
Python 3.12.4 (main, Jun  6 2024, 18:26:44) [GCC 13.3.0] on linux

Title: Python Applications vs. Libraries and Ad-hoc Environments
Summary
This section discusses the distinction between Python applications and libraries in Nixpkgs, explaining how applications are found throughout the tree while libraries are organized in separate sets per interpreter version. It details how applications are installed using `nix-env` or `environment.systemPackages` in NixOS, and emphasizes that Python libraries for development should be managed using `python.buildEnv` or `python.withPackages` to avoid import errors. It demonstrates creating an ad-hoc Python environment using `nix-shell` with specific packages like `numpy` and `toolz`, illustrating that only the specified packages and their dependencies are available in the interpreter's scope.