Home Explore Blog CI



nixpkgs

3rd chunk of `pkgs/by-name/README.md`
066a984e3ffe0a0a5cf223cd39ff30f15774f6bcc117b47a0000000100000cfa
There's some limitations as to which packages can be defined using this structure:

- Only packages defined using `pkgs.callPackage`.
  This excludes packages defined using `pkgs.python3Packages.callPackage ...`.

  Instead:
  - Either change the package definition to work with `pkgs.callPackage`.
  - Or use the [category hierarchy](../README.md#category-hierarchy).

- Only top-level packages.
  This excludes packages for other package sets like `pkgs.pythonPackages.*`.

  Refer to the definition and documentation of the respective package set to figure out how such packages can be declared.

## Validation

CI performs [certain checks](https://github.com/NixOS/nixpkgs-vet?tab=readme-ov-file#validity-checks) on the `pkgs/by-name` structure.
This is done using the [`nixpkgs-vet` tool](https://github.com/NixOS/nixpkgs-vet).

You can locally emulate the CI check using

```
$ ./ci/nixpkgs-vet.sh master
```

See [here](../../.github/workflows/nixpkgs-vet.yml) for more info.

## Recommendation for new packages with multiple versions

These checks of the `pkgs/by-name` structure can cause problems in combination:
1. New top-level packages using `callPackage` must be defined via `pkgs/by-name`.
2. Packages in `pkgs/by-name` cannot refer to files outside their own directory.

This means that outside `pkgs/by-name`, multiple already-present top-level packages can refer to some common file.
If you open a PR to another instance of such a package, CI will fail check 1,
but if you try to move the package to `pkgs/by-name`, it will fail check 2.

This is often the case for packages with multiple versions, such as

```nix
{
  foo_1 = callPackage ../tools/foo/1.nix { };
  foo_2 = callPackage ../tools/foo/2.nix { };
}
```

The best way to resolve this is to not use `callPackage` directly, such that check 1 doesn't trigger.
This can be done by using `inherit` on a local package set:
```nix
{
  inherit
    ({
      foo_1 = callPackage ../tools/foo/1.nix { };
      foo_2 = callPackage ../tools/foo/2.nix { };
    })
    foo_1
    foo_2
    ;
}
```

While this may seem pointless, this can in fact help with future package set refactorings,
because it establishes a clear connection between related attributes.

### Further possible refactorings

This is not required, but the above solution also allows refactoring the definitions into a separate file:

```nix
{
  inherit (import ../tools/foo pkgs)
    foo_1 foo_2;
}
```

```nix
# pkgs/tools/foo/default.nix
pkgs: {
  foo_1 = callPackage ./1.nix { };
  foo_2 = callPackage ./2.nix { };
}
```

Alternatively using [`callPackages`](https://nixos.org/manual/nixpkgs/unstable/#function-library-lib.customisation.callPackagesWith)
if `callPackage` isn't used underneath and you want the same `.override` arguments for all attributes:

```nix
{
  inherit (callPackages ../tools/foo { })
    foo_1 foo_2;
}
```

```nix
# pkgs/tools/foo/default.nix
{
  stdenv
}: {
  foo_1 = stdenv.mkDerivation { /* ... */ };
  foo_2 = stdenv.mkDerivation { /* ... */ };
}
```

### Exposing the package set

This is not required, but the above solution also allows exposing the package set as an attribute:

```nix
{
  foo-versions = import ../tools/foo pkgs;
  # Or using callPackages
  # foo-versions = callPackages ../tools/foo { };

  inherit (foo-versions) foo_1 foo_2;
}
```

Title: Limitations, Validation, and Versioning Recommendations for Packages in `pkgs/by-name`
Summary
This section outlines the limitations on packages that can be defined using the `pkgs/by-name` structure, focusing on packages defined using `pkgs.callPackage` and top-level packages. It details how to handle packages not fitting this criteria. It also describes the CI validation process using `nixpkgs-vet` and provides instructions for local emulation. The section then offers recommendations for managing new packages with multiple versions, highlighting the potential conflicts between the requirement to use `pkgs/by-name` for new top-level packages and the restriction on referring to files outside their directory. It suggests solutions like using `inherit` on a local package set or refactoring definitions into a separate file, and optionally exposing the package set as an attribute.