Home Explore Blog CI



nixpkgs

27th chunk of `doc/languages-frameworks/python.section.md`
6647b5792410a773e9729c2eaa35dae8a82be78d536914a10000000100000f22
  0503](https://www.python.org/dev/peps/pep-0503/#normalized-names). This means
  that characters should be converted to lowercase and `.` and `_` should be
  replaced by a single `-` (foo-bar-baz instead of Foo__Bar.baz).
  If necessary, `pname` has to be given a different value within `fetchPypi`.
* It's generally preferable to fetch `src` directly from the repo and not from
  PyPI. Use `fetchPypi` when there's a clear technical reason to do so.
* Packages from sources such as GitHub and GitLab that do not exist on PyPI
  should not use a name that is already used on PyPI. When possible, they should
  use the package repository name prefixed with the owner (e.g. organization) name
  and using a `-` as delimiter.
* Attribute names in `python-packages.nix` should be sorted alphanumerically to
  avoid merge conflicts and ease locating attributes.
* Non-python runtime dependencies should be added via explicit wrapping or
  patching (using e.g. `substituteInPlace`), rather than through propagation via
  `dependencies`/`propagatedBuildInputs`, to reduce clutter in `$PATH`.

This list is useful for reviewers as well as for self-checking when submitting packages.

## Package set maintenance {#python-package-set-maintenance}

The whole Python package set has a lot of packages that do not see regular
updates, because they either are a very fragile component in the Python
ecosystem, like for example the `hypothesis` package, or packages that have
no maintainer, so maintenance falls back to the package set maintainers.

### Updating packages in bulk {#python-package-bulk-updates}

A tool to bulk-update numerous Python libraries is available in the
repository at `maintainers/scripts/update-python-libraries`.

It can quickly update minor or major versions for all packages selected
and create update commits, and supports the `fetchPypi`, `fetchurl` and
`fetchFromGitHub` fetchers. When updating lots of packages that are
hosted on GitHub, exporting a `GITHUB_API_TOKEN` is highly recommended.

Updating packages in bulk leads to lots of breakages, which is why a
stabilization period on the `python-updates` branch is required.

If a package is fragile and often breaks during these bulks updates, it
may be reasonable to set `passthru.skipBulkUpdate = true` in the
derivation. This decision should not be made on a whim and should
always be supported by a qualifying comment.

Once the branch is sufficiently stable it should normally be merged
into the `staging` branch.

An exemplary call to update all python libraries between minor versions
would be:

```ShellSession
$ maintainers/scripts/update-python-libraries --target minor --commit --use-pkgs-prefix pkgs/development/python-modules/**/default.nix
```

## CPython Update Schedule {#python-cpython-update-schedule}

With [PEP 602](https://www.python.org/dev/peps/pep-0602/), CPython now
follows a yearly release cadence. In nixpkgs, all supported interpreters
are made available, but only the most recent two
interpreters package sets are built; this is a compromise between being
the latest interpreter, and what the majority of the Python packages support.

New CPython interpreters are released in October. Generally, it takes some
time for the majority of active Python projects to support the latest stable
interpreter. To help ease the migration for Nixpkgs users
between Python interpreters the schedule below will be used:

| When | Event |
| --- | --- |
| After YY.11 Release | Bump CPython package set window. The latest and previous latest stable should now be built. |
| After YY.05 Release | Bump default CPython interpreter to latest stable. |

In practice, this means that the Python community will have had a stable interpreter
for ~2 months before attempting to update the package set. And this will
allow for ~7 months for Python applications to support the latest interpreter.

Title: Python Package Maintenance, Bulk Updates, and CPython Update Schedule
Summary
This section covers Python package set maintenance in nixpkgs. It discusses normalizing names according to PEP 503, fetching sources directly from the repository, naming packages not on PyPI, sorting attribute names, and avoiding propagation of non-Python runtime dependencies. It then describes a tool for bulk-updating Python libraries and the need for a stabilization period. It details when to skip bulk updates for fragile packages. Finally, it outlines the CPython update schedule, including when to bump the package set window and the default interpreter, to balance being up-to-date and supporting the majority of Python packages.