Home Explore Blog Models CI



nixpkgs

3rd chunk of `doc/using/overlays.chapter.md`
42e4da55fd6d5a996af44c6ac4691818d6b8181dd0ff2e8b0000000100000f4f
    The AMD fork of the BLIS library, with attribute `amd-blis`, extends BLIS with optimizations for modern AMD CPUs. The changes are usually submitted to the upstream BLIS project after some time. However, AMD BLIS typically provides some performance improvements on AMD Zen CPUs. The complementary AMD LIBFLAME library, with attribute `amd-libflame`, provides a LAPACK implementation.

Introduced in [PR #83888](https://github.com/NixOS/nixpkgs/pull/83888), we are able to override the `blas` and `lapack` packages to use different implementations, through the `blasProvider` and `lapackProvider` argument. This can be used to select a different provider. BLAS providers will have symlinks in `$out/lib/libblas.so.3` and `$out/lib/libcblas.so.3` to their respective BLAS libraries.  Likewise, LAPACK providers will have symlinks in `$out/lib/liblapack.so.3` and `$out/lib/liblapacke.so.3` to their respective LAPACK libraries. For example, Intel MKL is both a BLAS and LAPACK provider. An overlay can be created to use Intel MKL that looks like:

```nix
final: prev:

{
  blas = prev.blas.override { blasProvider = final.mkl; };

  lapack = prev.lapack.override { lapackProvider = final.mkl; };
}
```

This overlay uses Intel's MKL library for both BLAS and LAPACK interfaces. Note that the same can be accomplished at runtime using `LD_LIBRARY_PATH` of `libblas.so.3` and `liblapack.so.3`. For instance:

```ShellSession
$ LD_LIBRARY_PATH=$(nix-build -A mkl)/lib${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH nix-shell -p octave --run octave
```

Intel MKL requires an `openmp` implementation when running with multiple processors. By default, `mkl` will use Intel's `iomp` implementation if no other is specified, but this is a runtime-only dependency and binary compatible with the LLVM implementation. To use that one instead, Intel recommends users set it with `LD_PRELOAD`. Note that `mkl` is only available on `x86_64-linux` and `x86_64-darwin`. Moreover, Hydra is not building and distributing pre-compiled binaries using it.

To override `blas` and `lapack` with its reference implementations (i.e. for development purposes), one can use the following overlay:

```nix
final: prev:

{
  blas = prev.blas.override { blasProvider = final.lapack-reference; };

  lapack = prev.lapack.override { lapackProvider = final.lapack-reference; };
}
```

For BLAS/LAPACK switching to work correctly, all packages must depend on `blas` or `lapack`. This ensures that only one BLAS/LAPACK library is used at one time. There are two versions of BLAS/LAPACK currently in the wild, `LP64` (integer size = 32 bits) and `ILP64` (integer size = 64 bits). The attributes `blas` and `lapack` are `LP64` by default. Their `ILP64` version are provided through the attributes `blas-ilp64` and `lapack-ilp64`. Some software needs special flags or patches to work with `ILP64`. You can check if `ILP64` is used in Nixpkgs with `blas.isILP64` and `lapack.isILP64`. Some software does NOT work with `ILP64`, and derivations need to specify an assertion to prevent this. You can prevent `ILP64` from being used with the following:

```nix
{
  stdenv,
  blas,
  lapack,
  ...
}:

assert (!blas.isILP64) && (!lapack.isILP64);

stdenv.mkDerivation {
  # ...
}
```

### Switching the MPI implementation {#sec-overlays-alternatives-mpi}

All programs that are built with [MPI](https://en.wikipedia.org/wiki/Message_Passing_Interface) support use the generic attribute `mpi` as an input. At the moment Nixpkgs natively provides the following MPI implementations:

-   [Open MPI](https://www.open-mpi.org/) (default), attribute name
    `openmpi`

-   [MPICH](https://www.mpich.org/), attribute name `mpich`

-   [MVAPICH](https://mvapich.cse.ohio-state.edu/), attribute name `mvapich`

To provide MPI enabled applications that use `MPICH`, instead of the default `Open MPI`, use the following overlay:

```nix
final: prev:

{
  mpi = final.mpich;
}
```

Title: Nixpkgs Overlays for BLAS/LAPACK and MPI Provider Switching
Summary
This section details how to use Nixpkgs overlays to switch implementations for BLAS/LAPACK and MPI. For BLAS/LAPACK, overlays override the `blas` and `lapack` packages by setting `blasProvider` and `lapackProvider` arguments to the desired library (e.g., Intel MKL or reference implementations), which then creates symbolic links to the chosen libraries. Runtime switching via `LD_LIBRARY_PATH` is also mentioned. It clarifies the difference between LP64 (default) and ILP64 integer sizes for BLAS/LAPACK, noting how to check and assert against ILP64 in derivations. For MPI, programs use a generic `mpi` attribute, and overlays can switch between providers like Open MPI (default), MPICH, or MVAPICH by simply setting `mpi = final.<provider_attribute_name>`.