Nixpkgs File Set Design: Single Files, `fileFilter` API, and Strict Path Existence

because it's not limited to just what is possible at evaluation time with `builtins.path`. Operations such as moving and adding files would be supported. ### Single files File sets cannot add single files to the store, they can only import files under directories. Arguments: - (+) There's no point in using this library for a single file, since you can't do anything other than add it to the store or not. And it would be unclear how the library should behave if the one file wouldn't be added to the store: `toSource { root = ./file.nix; fileset = <empty>; }` has no reasonable result because returning an empty store path wouldn't match the file type, and there's no way to have an empty file store path, whatever that would mean. ### `fileFilter` takes a path The `fileFilter` function takes a path, and not a file set, as its second argument. - (-) Makes it harder to compose functions, since the file set type, the return value, can't be passed to the function itself like `fileFilter predicate fileset` - (+) It's still possible to use `intersection` to filter on file sets: `intersection fileset (fileFilter predicate ./.)` - (-) This does need an extra `./.` argument that's not obvious - (+) This could always be `/.` or the project directory, `intersection` will make it lazy - (+) In the future this will allow `fileFilter` to support a predicate property like `subpath` and/or `components` in a reproducible way. This wouldn't be possible if it took a file set, because file sets don't have a predictable absolute path. - (-) What about the base path? - (+) That can change depending on which files are included, so if it's used for `fileFilter` it would change the `subpath`/`components` value depending on which files are included. - (+) If necessary, this restriction can be relaxed later, the opposite wouldn't be possible ### Strict path existence checking Coercing paths that don't exist to file sets always gives an error. - (-) Sometimes you want to remove a file that may not always exist using `difference ./. ./does-not-exist`, but this does not work because coercion of `./does-not-exist` fails, even though its existence would have no influence on the result. - (+) This is dangerous, because you wouldn't be protected against typos anymore. E.g. when trying to prevent `./secret` from being imported, a typo like `difference ./. ./sercet` would import it regardless. - (+) `difference ./. (maybeMissing ./does-not-exist)` can be used to do this more explicitly. - (+) `difference ./. (difference ./foo ./foo/bar)` should report an error when `./foo/bar` does not exist ("double negation"). Unfortunately, the current internal representation does not lend itself to a behavior where both `difference x ./does-not-exists` and double negation are handled and checked correctly. This could be fixed, but would require significant changes to the internal representation that are not worth the effort and the risk of introducing implicit behavior.

This document details design aspects and limitations of the Nixpkgs file set abstraction. It clarifies that file sets are intended for directories, not single files, due to a lack of meaningful filtering and an ill-defined 'empty' result. The `fileFilter` function is analyzed; it accepts a path, not a file set, which complicates direct composition but enables future reproducible predicate properties like `subpath`. A workaround using `intersection` is mentioned. The document also covers strict path existence checking, where non-existent paths cause errors. This design prevents accidental typos, though it means conditional file removal requires explicit handling via `maybeMissing`. Challenges with 'double negation' in `difference` operations are acknowledged.