Nixpkgs File Set Library: Empty Directories and String Paths

It could return the entire directory structure unchanged, but with all files removed, which would not be what one would expect. Trying to have a filter function that also supports directories will lead to the question of: What should the behavior be if `./foo` itself is excluded but all of its contents are included? It leads to having to define when directories are recursed into, but then we're effectively back at how the `builtins.path`-based filters work. - `./.` represents all files in `./.` _and_ the directory itself, but not its subdirectories, meaning that at least `./.` will be preserved even if it's empty. In that case, `intersection ./. ./foo` should only include files and no directories themselves, since `./.` includes only `./.` as a directory, and same for `./foo`, so there's no overlap in directories. But intuitively this operation should result in the same as `./foo` – everything else is just confusing. - (+) This matches how Git only supports files, so developers should already be used to it. - (-) Empty directories (even if they contain nested directories) are neither representable nor preserved when coercing from paths. - (+) It is very rare that empty directories are necessary. - (+) We can implement a workaround, allowing `toSource` to take an extra argument for ensuring certain extra directories exist in the result. - (-) It slows down store imports, since the evaluator needs to traverse the entire tree to remove any empty directories - (+) This can still be optimized by introducing more Nix builtins if necessary ### String paths File sets do not support Nix store paths in strings such as `"/nix/store/...-source"`. Arguments: - (+) Such paths are usually produced by derivations, which means `toSource` would either: - Require [Import From Derivation](https://nixos.org/manual/nix/unstable/language/import-from-derivation) (IFD) if `builtins.path` is used as the underlying primitive - Require importing the entire `root` into the store such that derivations can be used to do the filtering - (+) The convenient path coercion like `union ./foo ./bar` wouldn't work for absolute paths, requiring more verbose alternate interfaces: - `let root = "/nix/store/...-source"; in union "${root}/foo" "${root}/bar"` Verbose and dangerous because if `root` was a path, the entire path would get imported into the store. - `toSource { root = "/nix/store/...-source"; fileset = union "./foo" "./bar"; }` Does not allow debug printing intermediate file set contents, since we don't know the paths contents before having a `root`. - `let fs = lib.fileset.withRoot "/nix/store/...-source"; in fs.union "./foo" "./bar"` Makes library functions impure since they depend on the contextual root path, questionable composability. - (+) The point of the file set abstraction is to specify which files should get imported into the store. This use case makes little sense for files that are already in the store. This should be a separate abstraction as e.g. `pkgs.drvLayout` instead, which could have a similar interface but be specific to derivations. Additional capabilities could be supported that can't be done at evaluation time, such as renaming files, creating new directories, setting executable bits, etc.

This document continues the justification for design choices in the Nixpkgs file set library. It reiterates why empty directories are not explicitly representable, citing complications with filter functions and alignment with Git's file-centric model, while also acknowledging potential downsides like slower store imports and lack of direct representation for nested empty directories. The document also explains why file sets do not support Nix store paths as strings (e.g., `/nix/store/...`). It argues that such support would introduce complexities like requiring Import From Derivation (IFD), make path coercion verbose and potentially unsafe, hinder debugging, and lead to impure library functions. The core reasoning is that file sets are designed for specifying files to be imported *into* the store, not for manipulating files already *within* it, suggesting a separate abstraction like `pkgs.drvLayout` for the latter.