# lisp-modules {#lisp}

This document describes the Nixpkgs infrastructure for building Common Lisp
systems that use [ASDF](https://asdf.common-lisp.dev/) (Another System
Definition Facility). It lives in `pkgs/development/lisp-modules`.

## Overview {#lisp-overview}

The main entry point of the API are the Common Lisp implementation packages
themselves (e.g. `abcl`, `ccl`, `clasp-common-lisp`, `clisp`, `ecl`,
`sbcl`). They have the `pkgs` and `withPackages` attributes, which can be used
to discover available packages and to build wrappers, respectively.

The `pkgs` attribute set contains packages that were automatically
[imported](#lisp-importing-packages-from-quicklisp) from Quicklisp, and any
other [manually defined](#lisp-defining-packages-inside) ones. Not every package
works for all the CL implementations (e.g. `nyxt` only makes sense for `sbcl`).

The `withPackages` function is of primary utility. It is used to build
[runnable wrappers](#lisp-building-wrappers), with a pinned and pre-built
[ASDF FASL](#lisp-loading-asdf) available in the `ASDF` environment variable,
and `CL_SOURCE_REGISTRY`/`ASDF_OUTPUT_TRANSLATIONS` configured to
[find the desired systems on runtime](#lisp-loading-systems).

In addition, Lisps have the `withOverrides` function, which can be used to
[substitute](#lisp-including-external-pkg-in-scope) any package in the scope of
their `pkgs`. This will also be useful together with `overrideLispAttrs` when
[dealing with slashy systems](#lisp-dealing-with-slashy-systems), because they
should stay in the main package and be built by specifying the `systems`
argument to `build-asdf-system`.

## The 90% use case example {#lisp-use-case-example}

The most common way to use the library is to run ad-hoc wrappers like this:

`nix-shell -p 'sbcl.withPackages (ps: with ps; [ alexandria ])'`

Then, in a shell:

```
$ sbcl
* (load (sb-ext:posix-getenv "ASDF"))
* (asdf:load-system 'alexandria)
```

Also one can create a `pkgs.mkShell` environment in `shell.nix`/`flake.nix`:

```nix
let
  sbcl' = sbcl.withPackages (ps: [ ps.alexandria ]);
in
mkShell {
  packages = [ sbcl' ];
}
```

Such a Lisp can be now used e.g. to compile your sources:

```nix
{
  buildPhase = ''
    runHook preBuild

    ${sbcl'}/bin/sbcl --load my-build-file.lisp

    runHook postBuild
  '';
}
```

## Importing packages from Quicklisp {#lisp-importing-packages-from-quicklisp}

To save some work of writing Nix expressions, there is a script that imports all
the packages distributed by Quicklisp into `imported.nix`. This works by parsing
its `releases.txt` and `systems.txt` files, which are published every couple of
months on [quicklisp.org](https://beta.quicklisp.org/dist/quicklisp.txt).

The import process is implemented in the `import` directory as Common Lisp
code in the `org.lispbuilds.nix` ASDF system. To run the script, one can
execute `ql-import.lisp`:

```
cd pkgs/development/lisp-modules
nix-shell --run 'sbcl --script ql-import.lisp'
```

The script will:

1. Download the latest Quicklisp `systems.txt` and `releases.txt` files
2. Generate a temporary SQLite database of all QL systems in `packages.sqlite`
3. Generate an `imported.nix` file from the database

(The `packages.sqlite` file can be deleted at will, because it is regenerated
each time the script runs.)

The maintainer's job is to:

1. Re-run the `ql-import.lisp` script when there is a new Quicklisp release
2. [Add any missing native dependencies](#lisp-quicklisp-adding-native-dependencies) in `ql.nix`
3. For packages that still don't build, [package them manually](#lisp-defining-packages-inside) in `packages.nix`

Also, the `imported.nix` file **must not be edited manually**! It should only be
generated as described in this section (by running `ql-import.lisp`).

### Adding native dependencies {#lisp-quicklisp-adding-native-dependencies}

The Quicklisp files contain ASDF dependency data, but don't include native
library (CFFI) dependencies, and, in the case of ABCL, Java dependencies.