Overlay2: Container Read/Write Operations and Performance

type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/55f1e14c361b.../root, upperdir=/var/lib/docker/overlay2/l/ec444863a55a.../upper, workdir=/var/lib/docker/overlay2/l/ec444863a55a.../work) ``` The `rw` on the second line shows that the `overlay` mount is read-write. ## How container reads and writes work with `overlay2` <a name="how-container-reads-and-writes-work-with-overlay-or-overlay2"></a> ### Reading files Consider three scenarios where a container opens a file for read access with overlay. #### The file does not exist in the container layer If a container opens a file for read access and the file does not already exist in the container (`upperdir`) it is read from the image (`lowerdir`). This incurs very little performance overhead. #### The file only exists in the container layer If a container opens a file for read access and the file exists in the container (`upperdir`) and not in the image (`lowerdir`), it's read directly from the container. #### The file exists in both the container layer and the image layer If a container opens a file for read access and the file exists in the image layer and the container layer, the file's version in the container layer is read. Files in the container layer (`upperdir`) obscure files with the same name in the image layer (`lowerdir`). ### Modifying files or directories Consider some scenarios where files in a container are modified. #### Writing to a file for the first time The first time a container writes to an existing file, that file does not exist in the container (`upperdir`). The `overlay2` driver performs a `copy_up` operation to copy the file from the image (`lowerdir`) to the container (`upperdir`). The container then writes the changes to the new copy of the file in the container layer. However, OverlayFS works at the file level rather than the block level. This means that all OverlayFS `copy_up` operations copy the entire file, even if the file is large and only a small part of it's being modified. This can have a noticeable impact on container write performance. However, two things are worth noting: - The `copy_up` operation only occurs the first time a given file is written to. Subsequent writes to the same file operate against the copy of the file already copied up to the container. - OverlayFS works with multiple layers. This means that performance can be impacted when searching for files in images with many layers. #### Deleting files and directories - When a _file_ is deleted within a container, a _whiteout_ file is created in the container (`upperdir`). The version of the file in the image layer (`lowerdir`) is not deleted (because the `lowerdir` is read-only). However, the whiteout file prevents it from being available to the container. - When a _directory_ is deleted within a container, an _opaque directory_ is created within the container (`upperdir`). This works in the same way as a whiteout file and effectively prevents the directory from being accessed, even though it still exists in the image (`lowerdir`). #### Renaming directories Calling `rename(2)` for a directory is allowed only when both the source and the destination path are on the top layer. Otherwise, it returns `EXDEV` error

This section outlines how container reads and writes operate with the `overlay2` storage driver. It explains that reading a file involves checking the container layer (`upperdir`) first. If the file isn't there, it's read from the image (`lowerdir`). For writing, the first write to a file triggers a `copy_up` operation from the image to the container, after which changes are written to the container's copy. Deleting files creates a 'whiteout' file, and deleting directories creates an 'opaque directory' in the container layer, effectively hiding them without modifying the read-only image layer. Renaming a directory is allowed only when both the source and destination are on the top layer.