Home Explore Blog Models CI



nixpkgs

10th chunk of `doc/languages-frameworks/python.section.md`
c4b5601db5b520322e087e87b38d2fcb64bb44d860f22c9a0000000100000fd2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy; import toolz
```

Note that no other modules are in scope, even if they were imperatively
installed into our user environment as a dependency of a Python application:

```Python console
>>> import requests
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'requests'
```

We can add as many additional modules onto the `nix-shell` as we need, and we
will still get 1 wrapped Python interpreter. We can start the interpreter
directly like so:

```sh
$ nix-shell -p "python313.withPackages (ps: with ps; [ numpy toolz requests ])" --run python3
Python 3.13.3 (main, Apr  8 2025, 13:54:08) [GCC 14.2.1 20250322] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>>
```

Notice that this time it built a new Python environment, which now includes
`requests`. Building an environment just creates wrapper scripts that expose the
selected dependencies to the interpreter while re-using the actual modules. This
means if any other env has installed `requests` or `numpy` in a different
context, we don't need to recompile them -- we just recompile the wrapper script
that sets up an interpreter pointing to them. This matters much more for "big"
modules like `pytorch` or `tensorflow`.

Module names usually match their names on [pypi.org](https://pypi.org/), but
normalized according to PEP 503/508. (e.g. Foo__Bar.baz -> foo-bar-baz)
You can use the [Nixpkgs search website](https://nixos.org/nixos/packages.html)
to find them as well (along with non-python packages).

At this point we can create throwaway experimental Python environments with
arbitrary dependencies. This is a good way to get a feel for how the Python
interpreter and dependencies work in Nix and NixOS, but to do some actual
development, we'll want to make it a bit more persistent.

##### Running Python scripts and using `nix-shell` as shebang {#running-python-scripts-and-using-nix-shell-as-shebang}

Sometimes, we have a script whose header looks like this:

```python
#!/usr/bin/env python3
import numpy as np
a = np.array([1,2])
b = np.array([3,4])
print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
```

Executing this script requires a `python3` that has `numpy`. Using what we learned
in the previous section, we could startup a shell and just run it like so:

```ShellSession
$ nix-shell -p 'python313.withPackages (ps: with ps; [ numpy ])' --run 'python3 foo.py'
The dot product of [1 2] and [3 4] is: 11
```

But if we maintain the script ourselves, and if there are more dependencies, it
may be nice to encode those dependencies in source to make the script re-usable
without that bit of knowledge. That can be done by using `nix-shell` as a
[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)), like so:

```python
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p "python3.withPackages(ps: [ ps.numpy ])"
import numpy as np
a = np.array([1,2])
b = np.array([3,4])
print(f"The dot product of {a} and {b} is: {np.dot(a, b)}")
```

Then we execute it, without requiring any environment setup at all!

```sh
$ ./foo.py
The dot product of [1 2] and [3 4] is: 11
```

If the dependencies are not available on the host where `foo.py` is executed, it
will build or download them from a Nix binary cache prior to starting up, prior
that it is executed on a machine with a multi-user Nix installation.

This provides a way to ship a self bootstrapping Python script, akin to a
statically linked binary, where it can be run on any machine (provided nix is
installed) without having to assume that `numpy` is installed globally on the
system.

By default it is pulling the import checkout of Nixpkgs itself from our nix
channel, which is nice as it cache-aligns with our other package builds, but we
can make it fully reproducible by pinning the `nixpkgs` import:

```python
#!/usr/bin/env nix-shell
#!nix-shell -i python3 -p "python3.withPackages (ps: [ ps.numpy ])"

Title: Nixpkgs Python: Dynamic Environments, Reusable Modules, and Self-Bootstrapping Scripts
Summary
This chunk elaborates on managing Python environments with Nix, demonstrating that `nix-shell` creates isolated environments where only specified modules are available. It highlights Nix's efficient environment building, reusing binaries and recompiling only wrapper scripts, critical for large libraries like `pytorch`. It also covers finding Python module names in Nixpkgs (matching pypi.org). The text then introduces a powerful feature: embedding `nix-shell` commands directly into a Python script's shebang line. This allows scripts to declare dependencies directly in source, making them self-bootstrapping and portable. Nix then automatically fetches or builds dependencies, like a statically linked binary, allowing scripts to run on any Nix-enabled machine without global package installations. It concludes by mentioning `nixpkgs` pinning for full reproducibility.