OverlayFS Performance, Best Practices, and Limitations

handle `EXDEV` and fall back to a "copy and unlink" strategy. ## OverlayFS and Docker Performance `overlay2` may perform better than `btrfs`. However, be aware of the following details: ### Page caching OverlayFS supports page cache sharing. Multiple containers accessing the same file share a single page cache entry for that file. This makes the `overlay2` drivers efficient with memory and a good option for high-density use cases such as PaaS. ### Copyup As with other copy-on-write filesystems, OverlayFS performs copy-up operations whenever a container writes to a file for the first time. This can add latency into the write operation, especially for large files. However, once the file has been copied up, all subsequent writes to that file occur in the upper layer, without the need for further copy-up operations. ### Performance best practices The following generic performance best practices apply to OverlayFS. #### Use fast storage Solid-state drives (SSDs) provide faster reads and writes than spinning disks. #### Use volumes for write-heavy workloads Volumes provide the best and most predictable performance for write-heavy workloads. This is because they bypass the storage driver and don't incur any of the potential overheads introduced by thin provisioning and copy-on-write. Volumes have other benefits, such as allowing you to share data among containers and persisting your data even if no running container is using them. ## Limitations on OverlayFS compatibility To summarize the OverlayFS's aspect which is incompatible with other filesystems: [`open(2)`](https://linux.die.net/man/2/open) : OverlayFS only implements a subset of the POSIX standards. This can result in certain OverlayFS operations breaking POSIX standards. One such operation is the copy-up operation. Suppose that your application calls `fd1=open("foo", O_RDONLY)` and then `fd2=open("foo", O_RDWR)`. In this case, your application expects `fd1` and `fd2` to refer to the same file. However, due to a copy-up operation that occurs after the second calling to `open(2)`, the descriptors refer to different files. The `fd1` continues to reference the file in the image (`lowerdir`) and the `fd2` references the file in the container (`upperdir`). A workaround for this is to `touch` the files which causes the copy-up operation to happen. All subsequent `open(2)` operations regardless of read-only or read-write access mode reference the file in the container (`upperdir`). `yum` is known to be affected unless the `yum-plugin-ovl` package is installed. If the `yum-plugin-ovl` package is not available in your distribution such as RHEL/CentOS prior to 6.8 or 7.2, you may need to run `touch /var/lib/rpm/*` before running `yum install`. This package implements the `touch` workaround referenced above for `yum`. [`rename(2)`](https://linux.die.net/man/2/rename) : OverlayFS does not fully support the `rename(2)` system call. Your application needs to detect its failure and fall back to a "copy and unlink" strategy.

This section discusses OverlayFS performance in Docker, highlighting page caching benefits and copy-up operation overhead. It recommends using fast storage like SSDs and volumes for write-heavy workloads to bypass the storage driver's overhead. It also details OverlayFS limitations, particularly its incomplete POSIX standards implementation, which can cause issues with `open(2)` if read-only and read-write file descriptors point to different files due to copy-up. A workaround is to `touch` the files to force copy-up. The `rename(2)` system call is also not fully supported, requiring applications to fall back to a 'copy and unlink' strategy.